# 10 — Evaluation & Quality Measurement

Demonstrates how to measure search quality using the built-in evaluation metrics. This is how you answer "is my search actually good?" with numbers.

## What's new

- **ndcg()** — Normalized Discounted Cumulative Gain: measures ranking quality
- **mrr()** — Mean Reciprocal Rank: how high is the first correct result?
- **precisionAtK()** — what fraction of the top K results are relevant?
- **recallAtK()** — what fraction of relevant documents are in the top K?
- **RelevanceMap** — ground truth: which documents are relevant to each query?
- **Ablation** — measure the contribution of each signal layer

## Key concepts

### Relevance judgments

Before you can measure quality, you need ground truth: for each query, which documents are relevant and how relevant are they?

```ts
// Map<documentId, relevanceScore>
// 3 = highly relevant, 2 = relevant, 1 = marginally relevant
const expected = new Map([
  ["LEDGER", 3], // the correct answer
  ["ACCOUNT", 2], // related but not the primary answer
  ["COMBINED_PAYMENTS", 1], // tangentially related
]);
```

### What the metrics tell you

| Metric      | Question it answers                              | Range | Perfect                       |
| ----------- | ------------------------------------------------ | ----- | ----------------------------- |
| NDCG@K      | Are the best results near the top?               | 0-1   | 1.0                           |
| MRR         | How quickly do we find the first correct result? | 0-1   | 1.0 (first result is correct) |
| Precision@K | What fraction of top-K results are relevant?     | 0-1   | 1.0 (all relevant)            |
| Recall@K    | What fraction of relevant docs are in top K?     | 0-1   | 1.0 (all found)               |

NDCG is the most informative single metric — it accounts for both relevance grades and ranking position. MRR is simpler: it only cares about the first relevant result.

### When to use evaluation

- **After changing signals**: run your query set before and after — did NDCG improve?
- **After changing glossary**: did the new terms help or hurt?
- **Regression guard**: CI checks that NDCG stays above a threshold
- **Comparing approaches**: BM25-only vs hybrid vs graph-augmented

## Running

```bash
npx tsx examples/10-evaluation/main.ts
```