api: YAML JSON TOON Database
version: 1.0.0
format: yaml
dataset:
  id: 1651
  slug: rag-retrieval-metrics
  title: Retrieval Evaluation Metrics for RAG
  description: Retrieval metrics and diagnostic slices for evaluating the evidence selected before language-model generation.
  summary: Retrieval metrics and diagnostic slices for evaluating the evidence selected before language-model generation. Each row includes a practical application and a failure to check before using it.
  use_when:
    - Compare retrievers using a fixed judged query set.
    - Find why an answer lacks evidence before changing the generator.
  caveats:
    - Retrieval quality is separate from answer correctness.
    - Keep test queries out of retriever tuning and report uncertainty.
  related:
    - llm-evaluation-failure-modes
    - ml-evaluation-metrics
  reviewed_at: "2026-10-12"
  sources:
    - title: Stanford introduction to information retrieval
      url: "https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-in-information-retrieval-1.html"
    - title: Stanford ranked retrieval evaluation
      url: "https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-ranked-retrieval-results-1.html"
  reading:
    - evaluate-rag-retrieval-before-generation
  provenance: Original editorial reference notes by YJTOON, informed by the linked primary documentation. External specifications retain their own licenses.
  entry_count: 10
  fields:
    - section: data
      field: metric
      type: string
      example: Precision at k
    - section: data
      field: definition
      type: string
      example: Relevant retrieved items divided by k.
    - section: data
      field: use
      type: string
      example: Measure how much top-ranked context is useful.
    - section: data
      field: limitation
      type: string
      example: It depends on relevance judgments and the chose…
  category: AI & ML Terms
  category_slug: ai-ml-terms
  tags: ai-ml-terms,rag retrieval metrics,reference,engineering
  view_count: 14
  created_at: 1791798640
  updated_at: 1791798640
data:
  - metric: Precision at k
    definition: Relevant retrieved items divided by k.
    use: Measure how much top-ranked context is useful.
    limitation: It depends on relevance judgments and the chosen k.
  - metric: Recall at k
    definition: Relevant retrieved items divided by all known relevant items.
    use: Measure how much required evidence is recovered.
    limitation: Incomplete relevance labels distort the denominator.
  - metric: Hit rate at k
    definition: Fraction of queries with at least one relevant top-k item.
    use: Check whether the retriever finds any useful evidence.
    limitation: One relevant item can still be insufficient to answer.
  - metric: Mean reciprocal rank
    definition: Mean of the reciprocal rank of the first relevant result.
    use: Reward finding useful evidence early.
    limitation: It ignores relevant results after the first one.
  - metric: NDCG at k
    definition: Discounted graded relevance normalized by the ideal ranking.
    use: Compare rankings with multiple relevance grades.
    limitation: Judgment policy and gain function affect scores.
  - metric: Context coverage
    definition: Fraction of required answer facts supported by retrieved context.
    use: Find multi-document evidence gaps.
    limitation: It requires an explicit set of required facts.
  - metric: Duplicate rate
    definition: Fraction of retrieved context redundant with other selected items.
    use: Detect wasted context capacity.
    limitation: Overlapping chunks can be useful in some tasks.
  - metric: Retrieval latency
    definition: Elapsed time for retrieval under a declared load.
    use: Balance evidence quality against responsiveness.
    limitation: Cache state and concurrency change the distribution.
  - metric: Empty-result rate
    definition: Fraction of queries returning no eligible evidence.
    use: Track abstention and indexing problems.
    limitation: An empty result can be the correct behavior.
  - metric: Slice performance
    definition: A metric calculated by query or content subgroup.
    use: Expose failures hidden by an overall average.
    limitation: Small slices need uncertainty reporting.