# Recursive Language Models -- Comparison & Applicable Ideas

## Overview

This document records our analysis of [Recursive Language Models (RLMs)](https://arxiv.org/abs/2512.24601) (Zhang, Kraska, Khattab -- ICML 2026) against our schema discovery search service. RLMs are an inference-time paradigm enabling LLMs to process inputs up to 100x beyond their context window by recursively invoking themselves through a REPL environment.

We evaluated the pattern to identify transferable techniques, as we did with [LightRAG](../search/lightrag-comparison.md), [ChromaFs](../search/chromafs-comparison.md), [RAG-Anything](../search/raganything-comparison.md), [LLM Wiki](llm-wiki-comparison.md), [Stash](../memory/stash-comparison.md), and [Graphify](graphify-comparison.md).

> **Last updated:** 2026-05-10. Reference implementation at 4,196 stars (stable), 743 forks, 81 open issues. Last pushed Apr 27. Published on PyPI as `rlms`. iPython REPL with subprocess mode, 7 sandbox environments, trajectory visualizer (Node.js + shadcn/ui). Nobody has bridged RLM to retrieval -- closest attempts are toy repos. Our graph exploration RPCs (FindJoinPath, GetNodeNeighbors, GetNodeDetail) now implement idea #4 (REPL-style agent tools) from this comparison.

---

## How RLMs work

RLMs frame long-context processing as an inference-time scaling problem via three design principles:

1. **Symbolic handle to prompt**: The user prompt is stored as a REPL variable, not copied into context. The model manipulates it programmatically.
2. **Symbolic output construction**: Responses built in REPL variables rather than autoregressively generated.
3. **Symbolic recursion**: Code in the REPL can invoke the LLM via `llm_query()` inside loops.

### Results

| Task                       | Base GPT-5 | RLM(GPT-5) | Improvement |
| -------------------------- | ---------- | ---------- | ----------- |
| S-NIAH (needle search)     | 100%       | 100%       | --          |
| OOLONG (line aggregation)  | 44.0%      | 56.5%      | +28.4%      |
| OOLONG-Pairs (pairwise)    | 0.1%       | 58.0%      | +5700%      |
| BrowseComp+ (multi-hop QA) | 0.0%       | 91.3%      | N/A         |

---

## How our system differs

| Aspect            | RLMs                                 | @coda/search                                                 |
| ----------------- | ------------------------------------ | ------------------------------------------------------------ |
| **Problem**       | LLM can't see 10M tokens at once     | Agent needs to find relevant schema from 10K+ items in <15ms |
| **Layer**         | LLM inference                        | Retrieval                                                    |
| **Decomposition** | Recursive sub-calls on input chunks  | Multi-signal search fused via RRF                            |
| **Graph**         | None                                 | Explicit `LabeledGraph<T>` with typed edges, BFS/Dijkstra    |
| **Cost model**    | LLM calls per chunk per query (high) | Zero LLM calls at search time                                |
| **Latency**       | Seconds to minutes                   | 10-15ms                                                      |

---

## What we could adopt from RLM approaches

Four ideas that affect the search-to-agent boundary:

### 1. Task complexity-aware result depth (low effort, medium value)

Classify queries by estimated complexity and adjust result shape: O(1) queries (specific entity) get few results with full detail; O(n) queries (domain survey) get many results with summaries; O(n^2) queries (relational) get moderate results + join paths.

### 2. Structured result chunking (low effort, low value)

Group search results by structural affinity (domain, join paths) before returning to the agent.

### 3. Recursive search for multi-hop queries (medium effort, medium value)

A `DeepSearch` mode with a second retrieval pass seeded by first-pass results. The `ProximitySignal` already implements graph expansion; the gap is a second keyword pass with terms derived from first-pass results (pseudo-relevance feedback).

### 4. REPL-style agent tools for schema exploration (medium effort, high value)

Expose graph exploration as agent tools: `GetNeighbors(tableId, depth)`, `FindJoinPath(fromId, toId)`, `GetTableDetail(tableId)`.

**Status: DONE (PR-198)** -- `FindJoinPath` (bidirectional BFS), `GetNodeNeighbors` (depth-limited BFS with direction filter), `GetNodeDetail` (node metadata + glossary matches + degree) RPCs + agent tools (`find_join_path`, `get_table_neighbors`, `get_table_detail`).

---

## Enterprise readiness comparison

| Criterion                | RLMs                                                                | @coda/search                                            |
| ------------------------ | ------------------------------------------------------------------- | ------------------------------------------------------- |
| **Lifecycle management** | None -- each REPL session is ephemeral                              | Full state machine with abort coordination              |
| **Graceful degradation** | Fails if LLM API is unavailable                                     | Two-phase init, per-stage fallback, circuit breakers    |
| **Cost predictability**  | Unbounded LLM calls                                                 | Fixed infrastructure cost; zero LLM calls at query time |
| **Observability**        | REPL stdout/stderr logs                                             | 28 structured event types, OTel-compatible envelopes    |
| **Sandboxing**           | 7 environments (local, ipython, docker, modal, prime, daytona, e2b) | N/A (library, not execution environment)                |

---

## Developer experience comparison

| Aspect            | RLMs                                        | @coda/search                                                   |
| ----------------- | ------------------------------------------- | -------------------------------------------------------------- |
| **API surface**   | `rlm.completion(prompt)` -- single call     | 9 sub-path exports with focused public APIs                    |
| **Type safety**   | None -- Python REPL with dynamic types      | Full TypeScript generics                                       |
| **Composability** | Monolithic -- LLM controls everything       | Interface-based DI with 12 pluggable interfaces                |
| **Testing**       | Paper evaluations on 4 benchmarks           | 1,386 tests, Vitest with coverage, benchmark regression guards |
| **Debugging**     | Trajectory visualizer (Node.js + shadcn/ui) | Event bus with OTel-compatible traces, admin UI                |

---

## Abstraction quality comparison

| Principle                 | RLMs                                   | @coda/search                                                               |
| ------------------------- | -------------------------------------- | -------------------------------------------------------------------------- |
| **Single responsibility** | REPL is both execution and state store | Separated: SearchEngine / HybridSearch / SearchPipeline / Graph / EventBus |
| **Interface segregation** | One interface: `llm_query()`           | Focused interfaces: SchemaFetcher, DocumentTransformer, GraphBuilder, etc. |
| **Dependency inversion**  | Tightly coupled to specific LLM API    | All external deps injected via interfaces                                  |
| **Null object pattern**   | N/A                                    | `embeddingProvider: null` triggers keyword-only degraded mode              |

---

## Scope & limitations

### RLM limitations

- Not a retrieval system -- no indexing, ranking, or persistence
- Cost scales proportionally to input size per query
- Latency floor of seconds (multiple REPL round-trips)
- Emergent strategies vary between runs -- no reproducibility guarantees
- Primarily solo-maintained (88 of 92 commits from one author)

### @coda/search limitations

- Single-pass retrieval (no recursive decomposition)
- No semantic understanding beyond embedding similarity and glossary
- Embedding model coupling (snapshot vectors tied to model ID)

---

## Priority and sequencing

| Idea                            | Effort | Value  | Status                                  |
| ------------------------------- | ------ | ------ | --------------------------------------- |
| 1. Task complexity-aware depth  | Low    | Medium | Not started                             |
| 2. Structured result chunking   | Low    | Low    | After agent formatting layer stabilizes |
| 3. Recursive search (pass 2)    | Medium | Medium | After measuring retrieval gaps          |
| 4. REPL-style exploration tools | Medium | High   | **DONE (PR-198)**                       |

---

## Ecosystem

The reference implementation has grown to **4,196 stars** with 743 forks:

| Project               | Stars | Description                                                              |
| --------------------- | ----- | ------------------------------------------------------------------------ |
| **alexzhang13/rlm**   | 4,196 | Reference implementation. Published on PyPI as `rlms`. Last push Apr 27. |
| **AsyncReview**       | 441   | Agentic code review using RLM for unbounded-context analysis             |
| **fast-rlm**          | 282   | Optimized reimplementation                                               |
| **GenerateAgents.md** | 246   | DSPy + RLM integration for automated Agents.md                           |
| **rlm_repl**          | 237   | Alternative REPL implementation                                          |
| **rlm-cli**           | 176   | CLI wrapper                                                              |
| **rlm-claude-code**   | 90    | Claude Code integration with multi-provider routing                      |

**Key observation**: Nobody has bridged RLM patterns to retrieval/search. The pattern remains firmly in the inference layer.

---

## Verdict & recommendation

**One idea adopted, two remaining.** The highest-value transferable idea (REPL-style graph exploration tools) is implemented in PR-198. The remaining ideas (task complexity-aware depth, recursive search) are lower priority and should be pursued when retrieval gap analysis reveals specific multi-hop query failures.

---

## References

- Zhang, A. L., Kraska, T., Khattab, O.: [Recursive Language Models](https://arxiv.org/abs/2512.24601) (ICML 2026, accepted)
- Reference implementation: [github.com/alexzhang13/rlm](https://github.com/alexzhang13/rlm) (4,196 stars, MIT license)
- LightRAG comparison: [lightrag-comparison.md](../search/lightrag-comparison.md)
- LLM Wiki comparison: [llm-wiki-comparison.md](llm-wiki-comparison.md)