# txtai Comparison & Analysis

## Overview

This document compares [txtai](https://github.com/neuml/txtai) (v9.8.0, ~12,500 stars, Apache 2.0) against our schema discovery search library. txtai is one of the few systems that genuinely combines BM25 keyword search, vector similarity, and graph traversal in a single framework — making the fusion architecture comparison particularly relevant.

txtai is a **Python framework** (not a TypeScript library), so it is not a drop-in alternative. The comparison focuses on architectural patterns, fusion strategies, and feature coverage.

**Last updated**: 2026-05-10 (txtai v9.8.0, ~12,478 stars, 809 forks. Actively maintained — pushed May 10. Added MCP server support, agents via smolagents, Bayesian BM25 (BB25) normalization.)

---

## How txtai works

txtai's core is an **embeddings database** — a union of vector indexes, sparse scoring indexes, graph networks, and relational databases.

### Architecture layers

1. **Embeddings** — central abstraction wrapping dense ANN (FAISS default) + sparse BM25 + optional graph + optional SQL
2. **Pipelines** — modular NLP tasks (summarization, translation, LLM prompts)
3. **Workflows** — chain pipelines into DAGs with branching
4. **Agents** — autonomous problem-solving via HuggingFace smolagents
5. **API** — FastAPI server with REST endpoints + MCP support

### BM25 implementation

txtai has its own BM25 in pure Python/NumPy (`scoring/bm25.py`):

- Standard BM25 formula: `k1=1.2`, `b=0.75` (same defaults as ours)
- IDF: `log(1 + (N - df + 0.5) / (df + 0.5))`
- Additionally supports **BB25 (Bayesian BM25)** — sigmoid calibration that converts raw BM25 scores to calibrated probabilities using per-query adaptive parameters

### Hybrid search fusion

txtai supports **three fusion strategies**, auto-selected based on scoring config:

| Strategy               | When used                            | Method                                                              |
| ---------------------- | ------------------------------------ | ------------------------------------------------------------------- |
| Log-odds conjunction   | Bayesian (BB25) normalization active | Calibrated dense + sparse probabilities fused via weighted log-odds |
| Convex combination     | Scores are normalized (default)      | `score = w_dense * dense + w_sparse * sparse`                       |
| Reciprocal Rank Fusion | Scores are unnormalized              | Standard RRF: `sum(1/(rank+1) * weight)`                            |

**Key limitation**: Fusion is always exactly **2 signals** (dense + sparse). No multi-signal RRF across graph, fuzzy, glossary, or other signal types.

### Graph layer

Backed by NetworkX (default) or RDBMS:

- **Graph construction**: Edges between semantically similar documents (similarity-weighted)
- **Community detection**: Louvain, greedy modularity, async LPA for topic modeling
- **openCypher queries**: via GrandCypher library with `similar()` function for vector-in-graph queries
- **Entity extraction**: Optional LLM-driven knowledge graph construction
- **Graph algorithms**: Degree centrality, PageRank, shortest path, community detection
- **No structural graph from schema metadata** — graph is built from document similarity or LLM extraction, not FK/relationship inference

---

## How our system differs

| Aspect                 | txtai (v9.8.0)                                             | @coda/search                                                     |
| ---------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------- |
| **Language**           | Python 3.10+ (PyTorch required)                            | TypeScript, Node 24+                                             |
| **Purpose**            | General-purpose AI framework (search + pipelines + agents) | Schema discovery for AI agent tool selection                     |
| **Corpus**             | Any text documents                                         | Structured schema metadata (GraphQL types, Snowflake tables)     |
| **BM25**               | Standard BM25 + BB25 calibration                           | BM25+ with Porter stemming, camelCase splitting, prefix fallback |
| **Vector search**      | FAISS, hnswlib, Annoy, pgvector, sqlite-vec                | HNSW with uint8 quantization (4x memory reduction)               |
| **Fusion signals**     | 2 (dense + sparse)                                         | 10 (keyword×2 + vector + glossary + fuzzy + 5 graph signals)     |
| **Fusion strategies**  | Log-odds, convex combination, or RRF (auto-selected)       | RRF or WeightedSum (user-selected)                               |
| **Graph source**       | Document similarity or LLM extraction                      | Structurally derived (FK inference, type→field edges)            |
| **Graph signals**      | Degree, PageRank, community detection                      | Degree, adamic-adar, betweenness, column-density, proximity      |
| **Glossary expansion** | None                                                       | Domain glossary with fuzzy matching + synonym expansion          |
| **Fuzzy search**       | None built-in                                              | Edit-distance trie search via `FuzzyStage`                       |
| **Schema awareness**   | SQL metadata filtering (DuckDB)                            | FK inference, join path discovery, graph-augmented results       |
| **Reranking**          | Built-in cross-encoder pipeline                            | `RerankProvider` with deployed cross-encoder                     |
| **Query latency**      | ~10-50ms (search only, model not counted)                  | ~10-15ms end-to-end                                              |
| **Memory footprint**   | ~500MB+ (PyTorch + transformer model)                      | ~55MB for 10K quantized vectors + BM25 index                     |
| **Evaluation**         | None built-in                                              | NDCG/MRR/Recall with 189 golden queries, MRR regression guard    |
| **JS/TS integration**  | HTTP API client only (neuml/txtai.js)                      | Native TypeScript library                                        |
| **Deployment**         | Library, FastAPI server, Docker, txtai.cloud               | Embeddable library, deployed as Fargate microservice             |

---

## What we could adopt from txtai

### 1. Bayesian BM25 (BB25) score calibration

txtai's BB25 normalization converts raw BM25 scores to calibrated probabilities via per-query sigmoid parameters. This makes sparse scores directly comparable to dense scores without manual weight tuning.

**Applicability**: Medium. Our RRF fusion is rank-based and doesn't need score calibration, but if we ever switch to WeightedSumFusion in production, BB25 calibration would eliminate the score-scale mismatch between BM25 and cosine similarity.

### 2. Log-odds fusion

The Bayesian fusion strategy (Jeong, 2026) combines calibrated sparse and dense probabilities via weighted log-odds with confidence scaling. This is theoretically more principled than convex combination for 2-signal fusion.

**Applicability**: Low. Our 10-signal RRF handles heterogeneous signals well. Log-odds is designed for exactly 2 signals and assumes calibrated probabilities — it doesn't generalize to N signals.

### 3. openCypher graph queries

txtai supports openCypher queries (via GrandCypher) with a `similar()` function that integrates vector search into graph traversals. This allows queries like "find nodes similar to X that are connected to Y."

**Applicability**: Low. Our `FindJoinPath`, `GetNodeNeighbors`, `GetNodeDetail` RPCs serve the same agent use case with simpler, purpose-built APIs. openCypher adds flexibility we don't need.

### 4. MCP server support

txtai has built-in MCP (Model Context Protocol) support via fastapi-mcp. We already have a separate MCP server for the Coda agent.

**Applicability**: None — already addressed.

---

## Enterprise readiness comparison

| Dimension                | txtai (v9.8.0)                                         | @coda/search                                                       |
| ------------------------ | ------------------------------------------------------ | ------------------------------------------------------------------ |
| **Language / runtime**   | Python 3.10+, PyTorch required                         | TypeScript, Node 24+, ESM                                          |
| **Dependencies**         | Heavy: torch, transformers, faiss-cpu, huggingface-hub | Minimal: `@coda/data-structures`, `lru-cache`; ONNX for embeddings |
| **Memory footprint**     | ~500MB+ (transformer model)                            | ~55MB for 10K quantized vectors + BM25 index                       |
| **LLM dependency**       | Required for graph construction, optional for search   | None at query time; optional ONNX for embeddings                   |
| **State management**     | Implicit (save/load)                                   | Explicit state machine (CREATED→READY→DEGRADED→DESTROYED)          |
| **Graceful degradation** | Not documented                                         | Two-phase init, keyword-only fallback, per-stage fallback          |
| **Observability**        | Standard Python logging                                | 28 typed event types, OTel-compatible, per-query trace IDs         |
| **Test infrastructure**  | Unit tests, no retrieval benchmarks                    | ~1,386 tests, 189 golden queries, NDCG/MRR regression guard        |
| **Multi-tenancy**        | Separate embeddings instances                          | Per-tenant engine instances + SearchFilter                         |
| **Fusion signals**       | 2 (dense + sparse)                                     | 10 (keyword, expanded, vector, glossary, fuzzy, 5 graph signals)   |
| **Community**            | ~12.5K stars, 809 forks, commercial entity (NeuML)     | Internal project                                                   |

---

## Scope & limitations

### What txtai does that we don't need

1. NLP pipelines (summarization, translation, transcription, labeling)
2. Workflow DAG orchestration
3. Agent framework (smolagents)
4. 6 vector index backends (FAISS, hnswlib, Annoy, pgvector, sqlite-vec, scikit-learn)
5. LLM-driven entity extraction for knowledge graphs
6. openCypher graph queries
7. txtai.cloud hosted service

### What we do that txtai doesn't

1. 10-signal RRF fusion (vs 2-signal)
2. Schema-aware search with FK inference and graph-augmented results
3. 5 structural graph signals (degree, adamic-adar, betweenness, column-density, proximity)
4. Domain glossary expansion without LLM calls
5. Fuzzy matching via edit-distance trie search
6. Two-phase initialization with graceful degradation
7. Uint8-quantized HNSW (4x memory reduction vs float32)
8. Formal evaluation framework with NDCG/MRR regression guards
9. Per-signal score breakdown via `explain()` mode
10. Camel-case aware tokenization (critical for schema identifiers like `getUserById`)

---

## Verdict & recommendations

### If we started fresh, would we use txtai instead?

**No.** txtai is an impressive all-in-one Python AI framework, but it has fundamental mismatches with our requirements:

1. **Python-only** — no embeddable TypeScript library. The JS client is an HTTP wrapper requiring a Python server.
2. **2-signal fusion** — our 10-signal RRF provides significantly better ranking for schema queries where graph topology, glossary matches, and fuzzy corrections all contribute meaningfully.
3. **No schema awareness** — txtai treats documents as flat text. Our FK inference, join path discovery, and structural graph signals are fundamental to schema search quality.
4. **Heavy runtime** — PyTorch + transformer models require ~500MB+ memory. Our in-memory indexes use ~55MB.

### What are we doing better?

- **Signal depth**: 10 fused signals vs 2 — each one proven to improve NDCG on our benchmark
- **Schema specialization**: FK inference, join path discovery, column-density signals — purpose-built for the domain
- **Resource efficiency**: ~55MB vs ~500MB+, zero LLM dependency at query time
- **Type safety**: Full TypeScript generics vs Python type hints
- **Evaluation**: 189 golden queries with regression guards vs no built-in evaluation

### What is txtai doing better?

- **Fusion strategy diversity**: Three mathematically distinct fusion approaches (log-odds, convex, RRF) auto-selected by configuration. We offer two (RRF, WeightedSum) with manual selection.
- **BB25 calibration**: Bayesian score normalization eliminates the sparse/dense score-scale mismatch. Worth studying if we adopt WeightedSumFusion.
- **Breadth**: Search + NLP pipelines + workflows + agents in one framework. We focus solely on search.
- **Graph query language**: openCypher support enables ad-hoc graph queries beyond our fixed RPCs.
- **Community and commercial backing**: 12.5K stars + NeuML commercial entity + txtai.cloud.

### Overall assessment

txtai and @coda/search share the distinction of combining BM25 + vector + graph in a single system — a rare combination in the search landscape. The key architectural difference is that txtai fuses 2 signals (dense + sparse) with one of three strategies, while we fuse 10 signals via RRF. For schema discovery, where graph topology, glossary matches, and exact identifier matching all matter, our multi-signal approach produces measurably better results. txtai's BB25 calibration is the most interesting technique to study for potential adoption.

---

## References

- [txtai GitHub](https://github.com/neuml/txtai) — v9.8.0, ~12,478 stars, Apache 2.0
- [txtai Documentation](https://neuml.github.io/txtai/)
- [txtai.js](https://github.com/neuml/txtai.js) — JavaScript API client
- [Bayesian BM25 reference](https://neuml.github.io/txtai/methods/scoring/bm25/) — BB25 calibration
- LightRAG comparison: [lightrag-comparison.md](lightrag-comparison.md)
- Orama comparison: [orama-comparison.md](orama-comparison.md)