# 06 — Custom Pipeline

Builds on [03-custom-signal](../03-custom-signal/) by adding the two remaining extension points: query expansion and custom ranking stages.

## What's new

- **QueryExpander** — inject tokens into the query before ranking (synonym maps, abbreviation expansion)
- **SearchStage** — a custom ranking pass that runs concurrently with the built-in keyword/vector stages
- **QuerySignal** — a per-query signal computed from search candidates (requires vector search)

## Key concepts

### Pipeline flow

```
Query → [Expanders] → [Stages (concurrent)] → [Signals] → [RRF Fusion] → Results
```

1. **QueryExpanders** run first. Each returns expansion tokens that feed a separate `keyword_expanded` stage.
2. **SearchStages** run concurrently via `Promise.allSettled`. Built-in stages (keyword, keyword_expanded, vector) plus custom stages.
3. **Signals** (static + query) compute additional scores from document metadata or graph structure.
4. **RRF Fusion** combines all stage results and signal scores into a single ranked list.

### QueryExpander: synonym injection

A `QueryExpander` transforms the query by adding tokens. The original query tokens are unchanged — expansion tokens feed a separate keyword stage:

```ts
class SynonymExpander implements QueryExpander {
  readonly name = "synonyms";

  expand(query: string, _currentTokens: string[]): QueryExpansion {
    // "payout" → adds ["payment", "disbursement"]
    return { tokens };
  }
}
```

Result: query "payout" finds `PAYMENT` via the `keyword_expanded` signal even though "payout" appears nowhere in the corpus.

### SearchStage: column match

A `SearchStage` scores documents against the query context. Unlike BM25 (which treats everything as full-text), this stage does exact column-name matching:

```ts
class ColumnMatchStage implements SearchStage {
  readonly name = "column_match";
  readonly desc = true; // results pre-sorted descending

  async rank(context: StageContext): Promise<RankedEntry[]> {
    // Match context.tokens against column names
    // Return (id, score) entries
  }
}
```

The `StageContext` provides: `query`, `tokens`, `expansionTokens`, `itemCount`, `indexedIds`, and `limit`.

### QuerySignal: graph proximity

A `QuerySignal` is computed per-query using candidate IDs from the vector stage. In keyword-only mode, the vector stage is empty, so query signals don't fire. See [07-hybrid-search](../07-hybrid-search/) for an example where query signals participate in fusion.

## Running

```bash
npx tsx examples/06-custom-pipeline/main.ts
```

## Expected output

```
Indexed 5 tables

=== "payout" (synonym expansion) ===
  0.0385  PAYMENT  [keyword_expanded:#1]

=== "account_id" (column match stage) ===
  0.0755  ACCOUNT  [keyword:#1, column_match:#2]
  0.0729  CONTRACT  [keyword:#4, column_match:#1]
  0.0728  PAYMENT  [keyword:#2, column_match:#3]
  0.0702  VENDOR  [keyword:#3, column_match:#4]

=== "deal" (synonym + column match combined) ===
  0.0385  CONTRACT  [keyword_expanded:#1]
  0.0370  STATEMENT  [keyword_expanded:#2]

Note: QuerySignal (proximity) requires vector search.
See 07-hybrid-search for an example with EmbeddingProvider.
```