# Search Service Architecture

## Purpose & Audience

This document describes the architecture of the search microservice (`apps/search`). It is intended for engineers who need to understand, debug, extend, or operate the service. It assumes no prior knowledge of information retrieval concepts -- terms are defined in the glossary below.

For operational details (env vars, running locally, testing), see the [search README](../../apps/search/README.md). For the shared search library internals, see `packages/search/`.

---

## System Overview

The search service is a standalone ConnectRPC microservice that helps the AI agent discover relevant GraphQL operations and Snowflake tables from a natural-language query. Without it, the agent would need to scan thousands of schema items on every turn. The service indexes two data sources -- the federated GraphQL gateway and Snowflake's `ACCOUNT_USAGE` metadata -- and fuses keyword, vector, and domain-glossary signals into a single ranked result list.

```
                    ConnectRPC
  ┌───────────┐    searchGraphQL()    ┌───────────────────────────────────┐
  │  server   │───searchSnowflake()──>│           search service          │
  │ (Express) │    getSchema()        │                                   │
  └───────────┘<──────────────────────│  ┌─────────┐      ┌───────────┐   │
                                      │  │ GraphQL │      │ Snowflake │   │
       ┌──────────────────────────────│  │ Engine  │      │  Engine   │   │
       │                              │  └────┬────┘      └─────┬─────┘   │
       │                              │       │  SearchEngine    │         │
       │                              │       │  + Fetcher/Adapter│        │
       │    ┌─────────────┐           │  ┌────▼────────────────▼────┐     │
       │    │  S3 Bucket  │<─────────>│  │   BlobStore + SnapMgr    │     │
       │    └─────────────┘           │  └──────────────────────────┘     │
       │                              │  ┌─────────────────────────┐      │
       │    ┌─────────────┐           │  │  HuggingFace ONNX       │      │
       ├───>│  GraphQL    │<─────────>│  │  (embed + rerank)       │      │
       │    │  Gateway    │           │  └─────────────────────────┘      │
       │    └─────────────┘           └───────────────────────────────────┘
       │    ┌─────────────┐                        │
       └───>│  Snowflake  │<───────────────────────┘
            │ ACCOUNT_    │
            │ USAGE       │
            └─────────────┘
```

---

## Glossary

| Term                       | Definition                                                                                                                                                                                                           |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **BM25**                   | Best Matching 25 -- a keyword scoring formula that ranks documents by term frequency, inverse document frequency, and document length. The standard text-retrieval baseline.                                         |
| **HNSW**                   | Hierarchical Navigable Small World -- a graph-based approximate nearest-neighbor index. Supports O(log n) lookups by traversing a multi-layer proximity graph.                                                       |
| **RRF**                    | Reciprocal Rank Fusion -- a method for combining multiple ranked lists into one. Each document's score is the sum of `1/(k + rank)` across all lists it appears in.                                                  |
| **TF-IDF**                 | Term Frequency--Inverse Document Frequency -- a weighting scheme where terms that appear often in a document but rarely across the corpus score highest. BM25 extends this with length normalization and saturation. |
| **Embedding**              | A dense vector (e.g., 1024 floats) encoding the semantic meaning of a text. Similar texts produce similar vectors. Generated by a neural model.                                                                      |
| **Cross-encoder reranker** | A model that scores a (query, document) pair jointly, producing a more accurate relevance estimate than embedding similarity alone. Expensive -- used only on a small candidate set.                                 |
| **Quantization**           | Compressing float32 vectors to uint8 (1 byte per dimension instead of 4). Reduces memory and speeds up distance computation at the cost of minor precision loss.                                                     |
| **Score fusion**           | Combining relevance scores from multiple independent retrieval signals into a single ranking.                                                                                                                        |

---

## Architecture

The service uses a **SearchEngine + SchemaFetcher + focused interfaces** composition that separates lifecycle orchestration from data fetching and document transformation.

```
  SearchEngine<TRaw, TDoc>             SchemaFetcher<TRaw>          Focused interfaces
  ────────────────────────             ───────────────────          ──────────────────────────────
  Owns:                                Owns:                       DocumentTransformer (required):
  - init / refresh / destroy lifecycle - data fetching (fetch)       - raw → doc transform (adapt)
  - snapshot load/save                 - incremental diff (diff)     - keyword tokens (getKeywords)
  - ready / degraded flags             - connection teardown          - embed text (buildDocument)
  - abort coordination                                             GraphBuilder (optional):
  - hash tracking                                                    - graph construction (buildGraph)
  - HybridSearch + SearchPipeline                                  GlossaryProvider (optional):
                                                                     - glossary entries
```

**SearchEngine** is generic over `TRaw` (raw source type) and `TDoc` (searchable document type). It orchestrates the full lifecycle via four verbs: `init()` (fetch → adapt → index → embed), `refresh()` (diff → adapt → update), `search()` (delegate to pipeline), `destroy()` (abort, clear, release). Each data source gets its own engine instance with an independent snapshot key and `AbortController`. SearchEngine does not own polling — the caller decides when to call `refresh()` (timer, webhook, cron).

**SchemaFetcher** implementations know how to connect to a data source and retrieve its schema. `fetch()` returns the full schema on cold start; `diff()` returns incremental changes on subsequent polls.

The former monolithic `DocumentAdapter` has been split into three focused interfaces: **`DocumentTransformer`** (required -- raw-to-doc transform, keyword extraction, embed text), **`GraphBuilder`** (optional -- relationship graph construction), and **`GlossaryProvider`** (optional -- domain glossary entries). Each data source implements only the interfaces it needs. Selective indexing (which FQNs to include) is handled at fetch time by a `Filter` (implemented by `PrefixFqnFilter`), not at the adapter level.

Current implementations:

| Data source | SchemaFetcher      | Interfaces implemented                                    | TDoc                  | Graph type                             |
| ----------- | ------------------ | --------------------------------------------------------- | --------------------- | -------------------------------------- |
| GraphQL     | `GraphQLFetcher`   | `DocumentTransformer`, `GraphBuilder`, `GlossaryProvider` | `QueryFieldEntry`     | Type graph (returns/hasField edges)    |
| Snowflake   | `SnowflakeFetcher` | `DocumentTransformer`, `GraphBuilder`, `GlossaryProvider` | `SnowflakeTableEntry` | FK graph (inferred from `_ID` columns) |

Adding a new data source means implementing `SchemaFetcher<TRaw>` and `DocumentTransformer<TRaw, TDoc>` (plus optional `GraphBuilder`/`GlossaryProvider`), then registering them in `engine-factory.ts`. The engine handles lifecycle, indexing, snapshots, and search.

---

## Search Pipeline

Each query passes through a three-stage `SearchPipeline`:

```
  Query
    │
    ▼
  ┌──────────────────────────────────────┐
  │ Stage 1: Hybrid Search               │
  │  BM25 + HNSW + Glossary -> RRF fuse  │
  │  Over-fetches limit x 4 candidates   │
  └───────────────┬──────────────────────┘
                  │
                  ▼
  ┌──────────────────────────────────────┐
  │ Stage 2: Rerank (optional)           │
  │  Cross-encoder rescores candidates   │
  │  Sort by reranker score, slice top N │
  └───────────────┬──────────────────────┘
                  │
                  ▼
  ┌──────────────────────────────────────┐
  │ Stage 3: Graph Augmentation          │
  │  1-hop neighbors of top results      │
  │  Returned as `related` items         │
  └───────────────┬──────────────────────┘
                  │
                  ▼
  SearchPipelineResult { results, scores, related }
```

Each stage degrades gracefully: if the reranker fails, Stage 1 results pass through unchanged. If graph augmentation fails, `related` is empty. The pipeline never throws for search failures.

The `overFetch` multiplier (default 4) controls how many candidates Stage 1 returns before the reranker filters them down. Higher values improve reranker accuracy at the cost of latency.

---

## Hybrid Search Signals

`HybridSearch<T>` (in `@coda/search`) combines three independent retrieval signals, each implemented as a composable stage class:

| Stage        | Class                                     | Responsibility                                                                                                                                                 |
| ------------ | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Keyword**  | `KeywordStage`                            | InvertedIndex + StringTrie + BM25 scoring + prefix fallback + index mutations (`add`/`remove`/`clear`)                                                         |
| **Vector**   | `VectorStage`                             | Query embedding (LRU cache + dedup) + HNSW search + index mutations (`add`/`addFromSnapshot`/`remove`/`exportVectors`)                                         |
| **Glossary** | `GlossaryExpander` + `GlossaryMatchStage` | Split into two: `GlossaryExpander` (query expansion, runs before ranking) and `GlossaryMatchStage` (ranking stage, boosts documents matching glossary targets) |

Each stage owns both query-time ranking and index-time mutation, and can be tested independently. `HybridSearch` delegates all index operations to stages rather than reaching into their internals — it acts as a thin coordinator that owns the item map, candidate IDs, and orchestrates the stages. `HybridSearch.search()` orchestrates: run expanders, run stages, fuse signals via `ScoreFusion`, select top-K.

---

## Search Filter

`SearchFilter` allows callers to narrow results by document ID patterns. Applied inside `SearchPipeline.run()` between fusion (stage 1) and reranking (stage 2) -- the coarse-to-fine pattern.

```ts
interface SearchFilter {
  include?: string[]; // prefix patterns (e.g., "ANALYTICS.*") or exact IDs
  exclude?: string[]; // same syntax
  predicate?: (doc: unknown) => boolean; // arbitrary per-document filter
}
```

The filter uses prefix matching: `"ANALYTICS.*"` matches any ID starting with `"ANALYTICS."`. Graph augmentation respects the filter -- related items outside the filter are excluded.

---

## Data Sources

### GraphQL Strategy

**What it indexes:** Query fields and named types from the federated GraphQL gateway, obtained via introspection.

**Polling:** Introspects the gateway at a configurable interval (default 5 min). Compares the SHA-256 hash of the introspection result to detect changes. On change, computes a field-level diff (added/changed/removed query fields and types), re-embeds only changed items, and rebuilds the type graph.

**Graph augmentation:** A directed graph where query fields connect to their return types (`returns` edge) and types connect to their field types (`hasField` edge). Augmentation finds 1-hop neighbors of matched query-field nodes, surfacing related types.

**Glossary:** Receives domain glossary entries from `@coda/extensions` (`graphqlGlossaryEntries`), enabling term boosting and query expansion for GraphQL-specific business terms.

**Initialization:** Always single-phase (never degraded). On warm start, restores snapshot vectors and introspects the live schema. On cold start, introspects and embeds everything.

### Snowflake Strategy

**What it indexes:** Tables and columns from Snowflake's `INFORMATION_SCHEMA` / `ACCOUNT_USAGE`, including column names, types, comments, and `changedOn` timestamps.

**Selective indexing:** Only FQNs matching `SNOWFLAKE_ALLOWLIST` (and not in `SNOWFLAKE_BLOCKLIST`) are indexed and embedded. A `PrefixFqnFilter` (implementing the `Filter` interface) applies at catalog fetch time; tables outside the filter are never loaded.

**Polling:** Queries `ACCOUNT_USAGE` for tables changed since the last poll. Applies add/update/remove diffs incrementally.

**Graph augmentation:** A FK graph inferred from `<X>_ID` column naming conventions (e.g., `ARTIST_ID` implies a reference to a table named `ARTIST`). Same-schema targets are preferred, then same-database, then any. Augmentation is bidirectional.

**Two-phase initialization:** Phase 1 populates the keyword index and restores snapshot vectors immediately (service becomes ready, possibly degraded). Phase 2 embeds remaining items in a detached, abortable promise. This lets the service answer keyword-only queries within seconds while vector indexing completes in the background.

---

## Lifecycle

### Initialization

1. ONNX models load in parallel (embedding + optional reranker).
2. Circuit breakers wrap the embedding provider.
3. Each engine loads its snapshot via `SnapshotManager`, then calls `fetcher.fetch()`.
4. The adapter transforms raw items, the engine restores snapshot vectors, and populates indexes.
5. Engines report ready; the HTTP server starts accepting requests.

For Snowflake, initialization is two-phase: the engine marks itself as `degraded` while Phase 2 embedding runs. The health endpoint reports `degraded` (HTTP 200) instead of `ready`, and search returns keyword+glossary results only.

### Refresh (polling)

SearchEngine exposes a public `refresh()` method for incremental updates. The engine does not own a polling timer — the caller is responsible for scheduling refreshes (e.g., via `startPolling()` from `@coda/async`, webhooks, or cron). The engine guards against concurrent refreshes with internal deduplication.

Incremental diffs are fetched when `refresh()` is called. Diff operations:

- **add**: new documents embedded and inserted into both indexes
- **update**: existing documents re-embedded and replaced in both indexes
- **remove**: documents deleted from both indexes

### Graceful shutdown

On SIGTERM/SIGINT:

1. Stop all external polling timers.
2. Call `destroy()` on each engine — this aborts its `AbortController`, signals in-progress refreshes to stop, calls `fetcher.teardown()`, and awaits in-flight embed promises.
3. Wait for in-flight refreshes and embed promises to settle.
4. Close the HTTP server, dispose ONNX models.
5. Force-exit after `SHUTDOWN_TIMEOUT_MS` (default 10s) if drain stalls.

---

## Snapshot Persistence

Snapshots accelerate startup by persisting embedded vectors to S3, avoiding expensive re-embedding on restart.

**Format:** Gzip-compressed JSON. Float32 vectors are base64-encoded before JSON serialization to preserve bit-exact values. The `IndexSnapshot` structure:

```
{ version, hash, modelId, builtAt, documents[], vectors[] }
```

`vectors[i]` corresponds to `documents[i]` (parallel arrays). An empty `Float32Array` at index `i` means the document was not embedded.

**Key format:** `{prefix}v{SNAPSHOT_VERSION}/{segment}/{sanitized-model-id}/{timestamp}.json.gz`

Example: `search-snapshots/v1/graphql/mixedbread-ai--mxbai-embed-large-v1/2026-04-07T12-00-00.000Z.json.gz`

The key encodes the format version, embedding model, and timestamp. When the version or model changes, the key prefix changes and the service cold-starts with a fresh embed. Multiple snapshots are retained per engine; the `SnapshotManager` prunes old ones via a tiered retention policy.

**Versioning:** `SNAPSHOT_VERSION` (currently 1) and `modelId` are checked on load. Snapshots with a mismatched version or model are discarded, triggering a cold start.

**Retention:** `SnapshotManager` applies tiered pruning after each save via `computeRetention()`:

| Age        | Policy          |
| ---------- | --------------- |
| < 1 hour   | Keep all        |
| 1-24 hours | Keep 1 per hour |
| 1-7 days   | Keep 1 per day  |
| > 7 days   | Delete          |

**Vector storage:** `QuantizedHnswIndex` maintains an internal float32 store alongside its quantized HNSW graph. The engine exports vectors via `hybridSearch.toSnapshot()` when building snapshots.

**Storage layer:** `BlobStore` interface with implementations: `S3BlobStore` (production), `MemoryBlobStore` (tests), `NullBlobStore` (local dev without S3). `SnapshotManager` wraps `BlobStore` with versioned key generation, retention pruning, and event emission.

---

## Admin Service

The search service exposes a ConnectRPC `AdminService` alongside the main search RPCs, gated by role-based auth via ows-grass.

| Endpoint           | Type             | Description                                               |
| ------------------ | ---------------- | --------------------------------------------------------- |
| `GetEngines`       | Unary            | Engine status, doc/vector counts, poll timing             |
| `RefreshEngine`    | Unary            | Trigger manual poll for an engine                         |
| `TraceQuery`       | Unary            | Run a search with event tracing, return pipeline spans    |
| `GetGlossary`      | Unary            | Retrieve glossary entries for a source                    |
| `UpdateGlossary`   | Unary            | Update glossary entries (stub -- not yet implemented)     |
| `ListSnapshots`    | Unary            | List stored snapshots with timestamps and sizes           |
| `RollbackSnapshot` | Unary            | Restore a previous snapshot (stub -- not yet implemented) |
| `GetConfig`        | Unary            | Return runtime config                                     |
| `GetGraph`         | Unary            | Return graph topology (built on-demand from memory)       |
| `StreamEvents`     | Server-streaming | Live event feed from the EventBus                         |

Proto definitions are in `packages/search-api/proto/coda/search/v1/admin.proto`.

---

## Embedding & Reranking

### Embedding provider

Uses `@huggingface/transformers` ONNX runtime. The default model is `mixedbread-ai/mxbai-embed-large-v1` (1024 dimensions, CLS pooling). Asymmetric: document embeddings use raw text; query embeddings prepend a task-specific prefix.

Models are downloaded at Docker build time and cached in the image (see the `model-cache` Dockerfile stage). At runtime, ONNX loads from the local cache directory (`HF_CACHE_DIR`).

**Device:** CUDA by default in production (g4dn.xlarge with NVIDIA T4 GPU). Falls back to CPU when CUDA is unavailable. Controlled by `SEARCH_DEVICE`.

**Batching:** Large embed calls are split into configurable batch sizes (default 64 for the primary model). The event loop yields between batches to avoid blocking.

### Reranker

Optional cross-encoder model (no default — set `SEARCH_RERANKER_MODEL` to enable, e.g. `Xenova/bge-reranker-large`). Scores (query, document) pairs jointly for more accurate relevance estimates. If the reranker fails to load at startup, it is silently disabled.

### Circuit breakers

Both `embed()` and `embedQuery()` are wrapped in separate `CircuitBreaker` instances. Strategies: `consecutive`, `fixed` window, or `rolling` window (default). When open, embedding calls fail fast; the existing index continues serving keyword+glossary results.

States:

- **Closed** (normal): requests pass through. Failures are counted in a sliding window.
- **Open**: after `CB_FAILURE_THRESHOLD` failures in `CB_WINDOW_MS`, the circuit opens and requests fail fast.
- **Half-open**: after `CB_COOLDOWN_MS`, a probe request is allowed. On success (`CB_SUCCESS_THRESHOLD` successes), the circuit closes; on failure, it reopens.

When the circuit is open during a poll update, the update fails gracefully and the existing index continues serving queries.

---

## Health & Observability

**`GET /health`** -- Always returns 200. Used by container health checks.

**`GET /health/ready`** -- Returns the status of each index engine and both circuit breakers:

| Status        | HTTP | Meaning                                                         |
| ------------- | ---- | --------------------------------------------------------------- |
| `ready`       | 200  | All configured indexes are fully operational                    |
| `degraded`    | 200  | At least one index can serve queries (possibly keyword-only)    |
| `unavailable` | 503  | No indexes can serve queries, or both circuit breakers are open |

Response body includes per-index status (`ready` / `degraded` / `initializing` / `not configured`) and circuit breaker states (`closed` / `open` / `half-open`).

Structured JSON logging via pino is attached to every request (with request IDs). Sentry captures unhandled errors.

---

## Configuration Reference

Key environment variables grouped by concern. See the [search README](../../apps/search/README.md) for the full list with defaults and ranges.

| Group               | Variables                                                                                                                                                                                                                           | Notes                                           |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
| **Server**          | `PORT`, `ENVIRONMENT`, `SENTRY_DSN`, `SHUTDOWN_TIMEOUT_MS`                                                                                                                                                                          | Port defaults to 8081                           |
| **Models**          | `SEARCH_EMBEDDING_MODEL`, `SEARCH_RERANKER_MODEL`, `SEARCH_DEVICE`, `SEARCH_EMBEDDING_DTYPE`, `SEARCH_RERANKER_DTYPE`, `HF_CACHE_DIR`                                                                                               | Omit reranker model to disable reranking        |
| **GraphQL**         | `GRAPHQL_GATEWAY_URL`, `GRAPHQL_POLL_INTERVAL_MS`, `GRAPHQL_POLL_JITTER_MAX_MS`, `GRAPHQL_INTROSPECT_TIMEOUT_MS`                                                                                                                    | Omit gateway URL to disable GraphQL indexing    |
| **Snowflake**       | `SNOWFLAKE_ACCOUNT`, `SNOWFLAKE_USER`, `SNOWFLAKE_ROLE`, `SNOWFLAKE_WAREHOUSE`, `SNOWFLAKE_ALLOWLIST`, `SNOWFLAKE_BLOCKLIST`, `SNOWFLAKE_ENV`, `SNOWFLAKE_POLL_INTERVAL_MS`, `SNOWFLAKE_POLL_JITTER_MAX_MS`, `SNOWFLAKE_TIMEOUT_MS` | Omit account to disable Snowflake indexing      |
| **Snapshots**       | `S3_SNAPSHOT_BUCKET`, `S3_SNAPSHOT_PREFIX`, `S3_REGION`, `S3_ENDPOINT`                                                                                                                                                              | Omit bucket for NullSnapshotStore               |
| **Tuning**          | `RRF_K`, `MAX_SEARCH_LIMIT`, `MAX_QUERY_LENGTH`, `MAX_COLUMNS_PER_TABLE`                                                                                                                                                            | RRF_K range 1-100                               |
| **Circuit breaker** | `CB_STRATEGY`, `CB_FAILURE_THRESHOLD`, `CB_WINDOW_MS`, `CB_COOLDOWN_MS`, `CB_SUCCESS_THRESHOLD`                                                                                                                                     | Strategy: consecutive, fixed, rolling (default) |
| **Rate limit**      | `RATE_LIMIT_WINDOW_MS`, `RATE_LIMIT_MAX`                                                                                                                                                                                            | Per-IP rate limiting                            |

---

## Algorithms

This section provides a deep dive into the search algorithms and ranking system.

### BM25 keyword scoring

Okapi BM25 with parameters k1=1.2, b=0.75 (standard TREC values).

BM25 ranks documents by term frequency weighted by inverse document frequency and normalized for document length. Documents that share exact or stemmed tokens with the query score higher. Scoring is computed in-memory over an `InvertedIndex` that tracks per-document term frequencies and document lengths.

A prefix-match fallback via a trie index catches partial matches when exact keyword results are sparse.

#### Tokenization pipeline

Applied identically to queries and document text at index time:

1. **camelCase splitting** — `getUserById` → `get`, `user`, `by`, `id`
2. **Lowercase** — normalizes all tokens
3. **Stop word filter** — removes common English stop words (a, the, is, ...)
4. **Porter stemming** — `contracts` → `contract`, `running` → `run`

This ensures that a query for "contract" matches documents containing "contracts" or "contracting".

#### Dual-level keyword weighting

Inspired by LightRAG's local/global retrieval, keyword scoring is split into two RRF signals:

- **`keyword`** — raw query tokens (specific/low-level, e.g., "artist_id")
- **`keyword_expanded`** — glossary-expanded tokens only (conceptual/high-level, e.g., "royalties" when the query mentions "payments")

`KeywordStage.rank()` is called twice, and both signals enter RRF fusion independently. This gives specific entity matches stronger ranking power while still surfacing conceptually related items through glossary expansion.

### HNSW approximate nearest neighbor

Documents are embedded into dense vectors via the configured HuggingFace model. A `QuantizedHnswIndex` stores uint8-quantized vectors in the HNSW graph while retaining exact float32 vectors in an internal store for snapshot fidelity. Query vectors are quantized once per search call. The stage owns an LRU query cache and deduplicates concurrent embedding requests for the same query.

#### HNSW internals

HNSW builds a multi-layer proximity graph. At each layer, a node is connected to its `m` nearest neighbors. Queries start at the top layer and greedily descend, using `efSearch` to maintain a candidate set at each layer.

- **Construction**: each new document is inserted and linked to its nearest neighbors in the graph. `m` controls the number of bi-directional links per node (default 16); `efConstruction` controls search width during insertion (default 200). Higher values improve recall at the cost of build time and memory.
- **Query**: `efSearch` controls the candidate list size during traversal (default 50). Higher values improve recall at the cost of query latency.
- **Cosine similarity**: vectors are normalized at embed time; dot product of normalized int8 vectors approximates cosine similarity.

**Tradeoffs:**

| Parameter        | Higher value                             | Lower value                             |
| ---------------- | ---------------------------------------- | --------------------------------------- |
| `m`              | Better recall, more memory, slower build | Worse recall, less memory, faster build |
| `efConstruction` | Better recall, slower insert             | Worse recall, faster insert             |
| `efSearch`       | Better recall, slower query              | Worse recall, faster query              |

Default values (`m=16`, `efConstruction=200`, `efSearch=50`) are appropriate for corpora of tens of thousands of documents on CPU.

#### Uint8 quantization

Quantizing float32 vectors to uint8 reduces memory by 4x at the cost of minor precision loss.

**Process:**

1. Compute per-vector min and scale (range = max - min)
2. Map float values linearly to the uint8 range `[0, 255]`
3. Store the quantized bytes along with min/scale for reconstruction during dot product

**Accuracy:** Mean error is less than 5% at 1024 dimensions, which is well within acceptable bounds for approximate nearest-neighbor search. Exact float32 vectors are retained in `QuantizedHnswIndex`'s internal store for snapshot round-trip fidelity, so quantization error does not compound across restarts.

### Glossary boosting

Domain-specific glossary files (in `packages/extensions/`) map business terms to schema identifiers.

When the query contains a glossary term (word-boundary match, case-insensitive, fuzzy within edit distance 2), the corresponding target identifiers are boosted in the result set. Primary targets receive boost weight 1.0; related targets receive 0.5.

`GlossaryExpander` (a `QueryExpander`) produces `expansionTokens` (glossary-added synonyms) separate from the raw query tokens, enabling dual-level keyword weighting (see above). `GlossaryMatchStage` (a `SearchStage`) boosts documents whose IDs match glossary targets.

Glossary terms are also injected into search document text at index build time (via `context` fields), enriching the BM25 and vector indexes with domain vocabulary.

#### Glossary JSON format

Two glossary files in `packages/extensions/`:

- `graphql/graphql-glossary.json` — entries with GraphQL type/field names as targets
- `snowflake/snowflake-glossary.json` — entries with `DATABASE.{{env}}.TABLE` targets (resolved via `resolveGlossaryVars` at load time)

Each entry:

```jsonc
{
  "terms": ["royalties", "royalty"], // trigger terms (word-boundary, fuzzy matched)
  "targets": ["abacusContract"], // primary identifiers — boost weight 1.0
  "related": ["AbacusContractType"], // secondary identifiers — boost weight 0.5
  "context": "Royalties in Abacus...", // appended to search document text
  "domain": "royalties", // grouping label (informational)
  "priority": 2, // numeric multiplier: 0 = no boost, 1 = normal (default), 2 = double boost
}
```

#### Matching

At query time:

1. Tokenize and stem the query
2. Check each token against glossary terms (word-boundary match, case-insensitive, `maxEditDistance=2`)
3. For each matched entry, expand the query to include target and related identifiers
4. Boost matched identifiers in the RRF fusion step

#### Query expansion

Matched glossary targets are injected into the BM25 keyword query so that documents containing those exact identifiers score higher, even if the original query used a synonym or domain term.

### N-signal RRF fusion

All signals — keyword, keyword_expanded, vector, glossary, degree, and any caller-injected extras — produce independent ranked lists fused via RRF:

```
score(doc) = SUM over signals i:  1 / (k + rank_i(doc))
```

Where `rank_i` is the 1-based position in signal `i`'s list and `k` defaults to 25. Documents appearing in multiple signals naturally score higher. The fused scores are passed to a top-K selection algorithm with keyset pagination support.

Fusion accepts N named signals via `NamedSignal[]` (each carrying a `name` and a ranked `entries` list), rather than a hardcoded 3-field interface. This makes it straightforward to add or remove signals without changing the fusion API.

**Why k=25?** The original RRF paper (Cormack et al., 2009) recommends k=60 for homogeneous fusion (many systems of the same type). For heterogeneous fusion of 3-5 distinct signals, k=25 better rewards items that rank highly in multiple signals. This is the standard default in hybrid dense+sparse pipelines.

### Graph degree boost

Strategies provide `StaticSignal<T>[]` and `QuerySignal<T>[]` arrays on `HybridSearchConfig`. Static signals are computed once and cached until `invalidateStaticSignals()` is called -- typically after a poll rebuilds the graph. Query signals are computed fresh per search using top-ranked candidates as BFS anchors.

Five built-in signals participate in RRF fusion:

- **DegreeSignal** (static) — normalizes node degree to [0, 1]; highly-connected types/tables (hubs) get a ranking boost.
- **AdamicAdarSignal** (static) — sums `1/log(degree)` across neighbors; favors nodes with rare, specific neighbors over generic hub connections.
- **BetweennessSignal** (static) — Brandes' algorithm for betweenness centrality; "bridge" tables connecting separate domains score higher.
- **ColumnDensitySignal** (static, no graph) — documents with more keyword fields rank higher, distinguishing data-rich fact tables from small lookup tables.
- **ProximitySignal** (query-time) — multi-source BFS from top vector candidates boosts nearby graph neighbors.

This captures the intuition that a type returned by many query fields, or a table referenced by many FK columns, is more important -- and that items close to already-matched results are relevant.

### Graph augmentation

After the hybrid search pipeline returns top results, 1-hop graph neighbors are appended as `related` items.

#### Type graph (GraphQL)

A directed graph where nodes are GraphQL types and query fields. Edges:

- `qf:<name>` → `TypeName` (relation: `returns`) — query field to its return type
- `TypeName` → `FieldTypeName` (relation: `hasField`) — type to its field types

Augmentation adds the 1-hop neighbors of the matched query-field nodes. This surfaces related types when a specific query field matches.

#### FK graph (Snowflake)

Edges are inferred from `<X>_ID` column naming: a column named `ARTIST_ID` implies a reference to a table named `ARTIST`, `DIM_ARTIST`, or `FACT_ARTIST`. When multiple tables share the same name across databases and schemas, the same-schema target is preferred, then same-database, then any.

Augmentation is bidirectional: both the referencing table (has the `_ID` column) and the referenced table are included as neighbors of either endpoint.

### Weighted context allocation

Inspired by LightRAG's `pick_by_weighted_polling`, the `allocateBudget()` utility distributes detail across ranked results using a linear gradient. Top results get more context (e.g., all columns), lower results get less. The Snowflake handler uses this to vary column count per result.

See [LightRAG Comparison](../reference/comparisons/search/lightrag-comparison.md) for the full analysis of adopted patterns.

---

## Performance Characteristics

### Latency

| Operation                           | Time   | Notes                        |
| ----------------------------------- | ------ | ---------------------------- |
| Search query (in-memory)            | ~10 ms | BM25 + HNSW + RRF fusion     |
| Single query embedding              | ~5 ms  | ONNX inference (GPU)         |
| Reranking (40 candidates)           | ~50 ms | Cross-encoder inference      |
| Cold start (10K items)              | ~5 min | Dominated by embedding       |
| Warm start (10K items, 100 changed) | ~15 s  | Re-embeds only changed items |
| Poll diff (50 changed)              | ~3 s   | Re-embed + index merge       |

### Memory

The primary memory consumers are the float32 vector store and the HNSW graph. For 10K documents at 1024 dimensions:

- **Float32 vectors:** 10K x 1024 x 4 bytes = ~40 MB (internal to QuantizedHnswIndex)
- **Quantized HNSW graph:** 10K x 1024 x 1 byte + graph overhead = ~15 MB
- **Total vector memory:** ~55 MB

### Algorithmic complexity

| Component          | Time complexity     | Space complexity | Notes                                  |
| ------------------ | ------------------- | ---------------- | -------------------------------------- |
| BM25 index build   | O(n x d)            | O(n x d)         | n documents, d unique terms            |
| BM25 query         | O(q x df)           | O(1)             | q query terms, df = document frequency |
| HNSW insert        | O(m x log n)        | O(n x m)         | amortized                              |
| HNSW query         | O(efSearch x log n) | O(efSearch)      | approximate                            |
| RRF fusion         | O(k x n)            | O(n)             | k ranked lists                         |
| Glossary lookup    | O(q x g)            | O(g)             | g = glossary entries                   |
| Graph augmentation | O(degree)           | O(degree)        | 1-hop only                             |

### Cost

| Component               | Monthly cost |
| ----------------------- | ------------ |
| Fargate (2 vCPU / 4 GB) | ~$120        |
| S3 snapshots (~100 MB)  | < $1         |
| Network (VPC internal)  | < $1         |
| **Total**               | **~$120/mo** |

An in-process approach saves $50-200/month over managed alternatives (Elasticsearch, Pinecone, Weaviate) while providing sub-10 ms query latency.

---

## Tuning Guide

### RRF_K (rank weighting)

Controls how aggressively top-ranked results are weighted relative to lower-ranked ones.

- **Low k (e.g. 1-5)**: top rank dominates. Use when one signal is highly reliable.
- **Default k=25**: balanced. Works well when all three signals are equally trusted.
- **High k (e.g. 50-100)**: more uniform weighting across all candidates. Use when signals disagree frequently.

### overFetch (reranker quality)

The pipeline fetches `limit x overFetch` candidates before passing them to the reranker. Higher values give the reranker a richer pool to choose from but increase reranking latency.

- **Default 4x**: good balance for most query loads.
- **Increase to 8-10x** when reranker accuracy matters more than latency (e.g. batch indexing or low-QPS environments).

### efSearch (HNSW recall vs. latency)

Higher `efSearch` values produce better recall at the cost of longer query times. At corpus sizes typical for this service (thousands of documents), the default of 50 is usually sufficient. Increase if recall needs improvement; reduce if query latency is a concern.

---

## Decisions & Tradeoffs

### RRF over learned fusion

RRF is parameter-free (only k), requires no training data, and works well for 2-3 heterogeneous signals. Learned fusion would need labeled relevance data we do not have. `WeightedSumFusion` is available as an alternative for explicit weight tuning.

### HNSW over other ANN indexes

HNSW provides O(log n) lookups with tunable recall/latency tradeoffs. At our corpus size (thousands to tens of thousands), it outperforms simpler brute-force approaches without the complexity of more exotic indexes (IVF, ScaNN). The multi-layer graph structure handles incremental add/remove efficiently.

### In-process over external vector DB

At <50K documents, an in-process index avoids network latency (sub-ms vs. 20-200 ms), eliminates an infrastructure dependency, and costs $0 incremental. No managed vector DB natively supports our three-signal fusion with glossary boosting. Tradeoff: state is ephemeral and must be rebuilt from snapshots on restart.

### ConnectRPC over REST

ConnectRPC provides type-safe RPC with protobuf schemas, gzip/brotli compression, and works over standard HTTP/1.1 (compatible with ALB). The `@coda/search-api` package generates both server and client types from a single `.proto` definition.

### Two-phase Snowflake initialization

Snowflake catalogs can contain thousands of tables. Embedding all of them blocks startup for minutes. Two-phase init lets the service serve keyword+glossary queries immediately while vectors embed in the background. GraphQL does not use two-phase init because its corpus is smaller and embeds quickly.

---

## Deferred Work & Future Opportunities

- **Keyword search field-level boosting** -- `KeywordField` now supports per-field `weight` for BM25 scoring. Strategies could assign higher weights to table/field names vs. column names or comments to improve precision.
- **Branded FQN types** -- Snowflake FQNs (`DATABASE.SCHEMA.TABLE`) are plain strings. TypeScript branded types would prevent mixing FQNs with other string identifiers at compile time.
- **Eliminate parallel arrays in snapshots** -- The `documents[]` / `vectors[]` parallel-array format is fragile. Bundling each document with its vector in a single object would be safer, at the cost of more verbose serialization.
- **Bearer token auth** -- COD-82: the service currently has no authentication. Adding bearer token validation is planned.