api: YAML JSON TOON Database
version: 1.0.0
format: json
dataset:
  id: 1017
  slug: rag-retrieval-metrics
  title: Retrieval Evaluation Metrics for RAG
  description: Retrieval metrics and diagnostic slices for evaluating the evidence selected before language-model generation.
  summary: Retrieval metrics and diagnostic slices for evaluating the evidence selected before language-model generation. Each row includes a practical application and a failure to check before using it.
  use_when[2]: Compare retrievers using a fixed judged query set.,Find why an answer lacks evidence before changing the generator.
  caveats[2]: Retrieval quality is separate from answer correctness.,Keep test queries out of retriever tuning and report uncertainty.
  related[2]: llm-evaluation-failure-modes,ml-evaluation-metrics
  category: AI & ML Terms
  category_slug: ai-ml-terms
  tags: "ai-ml-terms,rag retrieval metrics,reference,engineering"
  view_count: 0
  created_at: 1791798640
  updated_at: 1791798640
  reviewed_at: 2026-10-12
  sources[2]{title,url}:
    Stanford introduction to information retrieval,"https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-in-information-retrieval-1.html"
    Stanford ranked retrieval evaluation,"https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-ranked-retrieval-results-1.html"
  reading[1]: evaluate-rag-retrieval-before-generation
  provenance: "Original editorial reference notes by YJTOON, informed by the linked primary documentation. External specifications retain their own licenses."
  entry_count: 10
  fields[4]{section,field,type,example}:
    data,metric,string,Precision at k
    data,definition,string,Relevant retrieved items divided by k.
    data,use,string,Measure how much top-ranked context is useful.
    data,limitation,string,It depends on relevance judgments and the chose…
data[10]{metric,definition,use,limitation}:
  Precision at k,Relevant retrieved items divided by k.,Measure how much top-ranked context is useful.,It depends on relevance judgments and the chosen k.
  Recall at k,Relevant retrieved items divided by all known relevant items.,Measure how much required evidence is recovered.,Incomplete relevance labels distort the denominator.
  Hit rate at k,Fraction of queries with at least one relevant top-k item.,Check whether the retriever finds any useful evidence.,One relevant item can still be insufficient to answer.
  Mean reciprocal rank,Mean of the reciprocal rank of the first relevant result.,Reward finding useful evidence early.,It ignores relevant results after the first one.
  NDCG at k,Discounted graded relevance normalized by the ideal ranking.,Compare rankings with multiple relevance grades.,Judgment policy and gain function affect scores.
  Context coverage,Fraction of required answer facts supported by retrieved context.,Find multi-document evidence gaps.,It requires an explicit set of required facts.
  Duplicate rate,Fraction of retrieved context redundant with other selected items.,Detect wasted context capacity.,Overlapping chunks can be useful in some tasks.
  Retrieval latency,Elapsed time for retrieval under a declared load.,Balance evidence quality against responsiveness.,Cache state and concurrency change the distribution.
  Empty-result rate,Fraction of queries returning no eligible evidence.,Track abstention and indexing problems.,An empty result can be the correct behavior.
  Slice performance,A metric calculated by query or content subgroup.,Expose failures hidden by an overall average.,Small slices need uncertainty reporting.