# LLM Wiki Pattern Comparison & Greenfield Analysis (Memory Layer) ## Overview This document records our analysis of [Karpathy's LLM Wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) (2026) against Coda's `@coda/memory` package and the server-side `MemoryService` integration. The LLM Wiki is an architectural pattern where LLMs incrementally build and maintain persistent wikis — structured, interlinked knowledge bases — rather than performing retrieval-augmented generation on-demand. Its core thesis: **knowledge should be compiled once and kept current, not re-derived on every query.** An earlier comparison evaluated the LLM Wiki against our [search service](../knowledge/llm-wiki-comparison.md), focusing on retrieval latency, glossary enrichment, and pre-compiled entity summaries. This comparison evaluates the pattern against a different layer: the **agent memory system**. Where the search comparison asked "can compiled knowledge improve schema discovery?", this comparison asks three deeper questions: 1. **Philosophy**: How does "compile once, keep current" relate to "accumulate observations, consolidate into facts"? 2. **Architecture**: Could the LLM Wiki pattern replace, complement, or inform `@coda/memory`'s observation-to-fact pipeline? 3. **Boundary**: Should "what the agent knows about the domain" and "what the agent has learned about the user" live in the same system? We evaluated the pattern to identify transferable techniques and assess architectural boundaries, as we did with [MemOS](memos-comparison.md) (memory OS), [Memori](memori-comparison.md) (agent memory infrastructure), [Stash](stash-comparison.md) (agent memory), and the other [search comparisons](../search/). --- ## How the LLM Wiki works The LLM Wiki is a three-layer architecture for building compounding knowledge bases. Karpathy's framing: "Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation." ### Layer 1 — Raw sources Immutable curated documents — articles, papers, data files. The LLM reads from them but never modifies them. These are the single source of truth. ### Layer 2 — The wiki A directory of LLM-generated markdown files: summaries, entity pages, concept pages, comparisons, synthesis documents. The LLM owns this layer entirely — creating, updating, and cross-referencing pages as sources are ingested. The wiki is the persistent, compounding artifact. Cross-references are pre-built. Contradictions are pre-flagged. Synthesis reflects everything ingested so far. ### Layer 3 — The schema A configuration document (e.g., `CLAUDE.md`) that instructs the LLM on structure, conventions, and workflows for ingestion, querying, and maintenance. Co-evolved between the human and LLM over time. ### Core operations **Ingestion**: A source is added and the LLM processes it — writes a summary page, updates the index, updates relevant entity and concept pages across the wiki, appends to the log. A single source might touch 10-15 pages. Ingestion is supervised (human reads summaries, guides emphasis) or batch-processed. **Querying**: The LLM reads `index.md` first to find relevant pages, drills into them, and synthesizes an answer with citations. Key insight: **good answers can be filed back into the wiki as new pages** — a comparison, analysis, or connection discovered during a query becomes a persistent artifact rather than disappearing into chat history. Knowledge compounds. **Linting**: Periodic health checks identify contradictions between pages, stale claims superseded by newer sources, orphan pages with no inbound links, important concepts lacking their own page, missing cross-references, and data gaps. The LLM suggests corrections and new sources. ### Navigation infrastructure **index.md**: Content-oriented catalog listing every page with a one-line summary, organized by category. Updated during each ingest. The LLM reads it first when answering queries. Works surprisingly well up to ~100 sources and hundreds of pages, avoiding embedding infrastructure. **log.md**: Append-only chronological record of ingests, queries, and maintenance with parseable timestamps (e.g., `## [2026-04-02] ingest | Article Title`). Provides a timeline of wiki evolution. ### Why the pattern works Karpathy: "The tedious part of maintaining a knowledge base is not the reading or the thinking — it's the bookkeeping. Updating cross-references, keeping summaries current, noting when new data contradicts old claims, maintaining consistency across dozens of pages. Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don't get bored, don't forget to update a cross-reference, and can touch 15 files in one pass." --- ## How our system works Coda's `@coda/memory` package (`packages/memory/`) provides cross-session learning through a two-stage pipeline: raw observations are consolidated into durable facts with confidence decay, then facts are injected into the system prompt at session start. ### Architecture ``` User Query │ ▼ emitQueryPatternObservations() ← domain keyword matching (before agent loop) │ ▼ Tool Execution │ ▼ emitMemoryForToolExecution() ← fire-and-forget, best-effort ├─ emitToolObservations() [tool_success / tool_failure] └─ emitEntityObservations() [account, product, schema table access] │ ▼ ObservationLog.append() ← sorted set, keyed by timestamp │ ▼ [Next session start] │ ▼ MemoryService.loadMemory(identityId) │ ▼ consolidate() 1. Read checkpoint (last-consolidated timestamp) 2. Fetch observations since checkpoint 3. Deduplicate to fact keys (one reinforcement per fact per session) 4. Reinforce existing facts or create new (confidence 0.5) 5. Decay unreinforced facts; delete if below 0.1 threshold 6. Persist updated facts + new checkpoint 7. Prune old observations (7-day buffer) │ ▼ formatMemorySection() ← ~300 token budget │ ▼ System prompt injection ← memory first, then domain instructions │ ▼ Tool ranking via toolAffinities ← confidence-weighted boost in searchCatalog() ``` ### Data model Two entity types: | Entity | Storage | Purpose | TTL | | --------------- | ------------------------------- | --------------------------------------------- | -------------------------------------- | | **Observation** | Redis sorted set (by timestamp) | Append-only raw events from tool execution | 7-day buffer after consolidation prune | | **Fact** | Redis hash map (by identity) | Consolidated beliefs with confidence 0.0--1.0 | Exponential decay; expires below 0.1 | Four observation types: `tool_success`, `tool_failure`, `entity_access`, `query_pattern`. Five fact categories: `tool_affinity`, `tool_avoidance`, `entity_frequency`, `domain_preference`, `failure_pattern`. ### Decay strategies Two pluggable `DecayStrategy` implementations behind a common interface: | Strategy | Formula | Half-life | When to use | | -------------------- | ----------------------------------------- | ----------------- | ------------------------------------ | | **ExponentialDecay** | `confidence * 0.95 ^ days` | ~13.5 days | Default; simple time-based decay | | **AdaptiveDecay** | `confidence * 0.95 ^ (days / easeFactor)` | ~17.5--33.75 days | SM-2 inspired; per-fact ease factors | ### System prompt injection `formatMemorySection()` ranks facts by `confidence * reinforcementCount`, groups by display category (Tool preferences, Frequent entities, Domain focus, Known issues), and assembles markdown under a "What you know about this user" header. Budget: ~300 tokens. Returns empty string if no facts pass the filter. ### Tool affinity integration `extractToolAffinities()` produces a `Map` from `tool_affinity` (positive) and `tool_avoidance` (negative) facts. Consumed by `searchCatalog()` as a tiebreaker boost (max +/-2 points). --- ## Side-by-side comparison | Aspect | LLM Wiki | Coda `@coda/memory` | | ------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | **Problem** | Knowledge re-derivation: LLMs rediscover answers from scratch every query | Agent amnesia: agent forgets user behavior between sessions | | **Knowledge type** | Domain/structural: "what is true about the world" | Behavioral/preference: "what the user tends to do" | | **Architecture** | Three-layer: raw sources -> compiled wiki -> schema/config | Two-stage pipeline: observations -> facts -> prompt injection | | **Core abstraction** | Wiki page: markdown document with cross-references and metadata | `Observation` + `Fact`: typed event -> consolidated belief with confidence | | **Compilation method** | LLM-driven: reads sources, synthesizes pages, maintains cross-references | Deterministic: counting, averaging, decaying. Zero LLM calls | | **Update trigger** | Explicit: human adds source or requests lint | Implicit: observations emitted automatically during tool execution | | **Knowledge persistence** | Git repo of markdown files (version-controlled, diffable) | Redis sorted sets + hash maps (or in-memory fallback) | | **Scope** | Shared: one wiki per domain, used by anyone | Per-user: facts scoped to `identityId`, never shared | | **Temporal model** | Append + update: pages are revised; log.md tracks changes chronologically | Decay: facts lose confidence without reinforcement; expire below threshold | | **Knowledge quality** | Maintained by linting: contradictions, staleness, orphans, gaps detected | Maintained by decay: unreinforced facts naturally expire | | **Retrieval** | Index-based: LLM reads `index.md`, navigates to pages, synthesizes | Bulk load: all facts loaded into ~300 tokens of system prompt | | **Feedback loop** | Explicit: query answers filed back as wiki pages; explorations compound | Implicit: tool success/failure reinforces existing observations | | **Ingestion cost** | High: LLM reads source + writes/updates 10-15 pages per source | Zero: observation emission is fire-and-forget string appends | | **Query cost** | High: LLM reads pages + synthesizes (seconds per query) | Zero: facts pre-loaded at session start; no per-query cost | | **Scale** | ~100 sources, hundreds of pages (`index.md` suffices) | ~50 facts per user, ~300 tokens of prompt | | **LLM dependency** | Required for all operations: ingestion, querying, linting | Zero -- all processing is deterministic aggregation | | **Human involvement** | Active: human curates sources, guides emphasis, reviews output | Passive: user behavior is observed automatically | | **Maintenance** | LLM-driven linting with human review | Automatic: decay + consolidation run on every session start | | **Access control** | File-system level (git permissions, directory structure) | `identityId`-keyed storage with prefixed Redis keys | | **Maturity** | Architectural pattern (2026); no reference implementation | Production-deployed; ~900 LoC with comprehensive test suite | --- ## Where the LLM Wiki pattern excels ### 1. Knowledge compilation and re-use The LLM Wiki's central innovation is that knowledge is compiled once and served repeatedly. When a source is ingested, the LLM extracts, synthesizes, and cross-references its contents into wiki pages. Every subsequent query reads compiled output — the synthesis, contradictions, and cross-references are already resolved. Our memory system does not compile knowledge. It compiles _behavioral statistics_: tool affinity counts, entity access frequency, domain keyword matches. The agent knows that the user _uses_ `query_snowflake` frequently and _accesses_ Account 12345 often, but it doesn't know that `query_snowflake` works best with specific join patterns, that Account 12345 is a major label with 15,000 contracts, or that "royalty" queries should check both `FACT_ROYALTY_PAYMENT` and `VW_REVENUE_BY_COUNTRY`. This is the fundamental gap: **our memory tracks what the user does but not what the domain means.** ### 2. Cross-referencing and relationship maintenance The LLM Wiki automatically maintains relationships between knowledge pages. When a new source mentions an entity already in the wiki, the LLM updates the entity's page, adds cross-references, and revises synthesis pages. Relationships are first-class artifacts. Our fact model is flat — facts have no relationships to each other. A `tool_affinity:query_snowflake` fact and an `entity_frequency:schema_table:FACT_ROYALTY_PAYMENT` fact are stored independently, even though they're semantically related (the user prefers `query_snowflake` _because_ they work with royalty payment tables). The system cannot reason about connections between facts. ### 3. Compounding knowledge through query feedback The LLM Wiki's query operation has a feedback loop: good answers can be filed back into the wiki as new pages. A comparison table, analysis, or connection discovered during a query becomes a permanent artifact. Knowledge compounds with every interaction. Our memory system has no query feedback. The observation pipeline captures tool execution events but not the _quality_ or _content_ of agent responses. If the agent produces an excellent analysis of royalty payment patterns, that analysis disappears into chat history. The next session starts from zero understanding of the domain. ### 4. Proactive quality maintenance through linting The LLM Wiki pattern includes periodic health checks that identify contradictions, stale claims, orphan pages, and coverage gaps. The system actively maintains knowledge quality. Our memory system relies entirely on temporal decay for quality maintenance. Facts that aren't reinforced naturally expire. This handles obsolescence (user no longer accesses Account 12345) but cannot detect inconsistency (two facts that contradict each other), coverage gaps (important entities with no memory), or staleness (a fact that's still reinforced but whose content is outdated). ### 5. Shared, versioned knowledge The LLM Wiki is a git repo of markdown files — version-controlled, diffable, inspectable by anyone. Changes can be reviewed, reverted, or branched. Multiple users share the same knowledge base. Our facts are per-user, stored in Redis with no version history. If a fact's content changes (e.g., entity name updated), the old content is overwritten with no audit trail. Facts cannot be shared across users — if two users work with the same accounts and tables, each builds their own independent fact set. --- ## Where our system excels ### 1. Zero LLM dependency Our consolidation pipeline is pure aggregation: counting, averaging, decaying. No LLM calls, no embedding, no external API dependencies. The LLM Wiki requires LLM calls for every operation — ingestion (read source + write/update 10-15 pages), querying (read pages + synthesize), and linting (detect contradictions + suggest fixes). Every knowledge operation costs money and introduces non-determinism. For a system where memory is best-effort enhancement (not core functionality), deterministic zero-cost processing is the right trade-off. ### 2. Automatic, passive knowledge accumulation Our system observes user behavior automatically during normal agent operation. No explicit ingestion step, no human curation, no source management. The LLM Wiki requires active human participation: curating sources, adding them to the collection, guiding the LLM's emphasis, reviewing wiki output. This is appropriate for a personal knowledge base but impractical for an enterprise agent serving many users concurrently. ### 3. Temporal relevance through confidence decay Our `DecayStrategy` interface with `ExponentialDecay` and `AdaptiveDecay` implementations provides principled temporal relevance management. Facts that aren't reinforced naturally age out. The LLM Wiki has no decay mechanism — wiki pages persist indefinitely unless explicitly linted and removed. For user behavioral data that genuinely changes over time (a user who shifts focus from royalties to contract analysis), automatic decay is essential. ### 4. Graceful degradation `MemoryService.loadMemory()` races consolidation against a 500ms timeout and returns empty results on any failure. Memory is strictly additive — if it's slow or broken, the agent falls back to stateless behavior with no degradation in core functionality. The LLM Wiki has no fallback model — if wiki pages are missing, stale, or corrupted, the LLM's answers degrade because they depend on compiled knowledge that doesn't exist or is wrong. ### 5. Per-user personalization Our system learns per-user behavioral patterns: which tools each user prefers, which entities they access, which domains they ask about. The LLM Wiki produces domain-level knowledge that is the same for every user. Both are valuable, but our system provides personalization that the wiki pattern does not address. ### 6. Budget-constrained prompt injection `formatMemorySection()` ranks facts by signal strength (`confidence * reinforcementCount`), groups by category, and truncates to a 300-token budget. Memory enhances the prompt without dominating it. The LLM Wiki pattern has no built-in prompt budget — wiki pages are variable-length artifacts that the consumer must manage. Integration into a constrained agent loop requires additional work. --- ## The fundamental distinction: knowledge vs memory The LLM Wiki and `@coda/memory` solve different problems at different layers, and understanding this distinction is critical to deciding whether they should live in the same system. ### Two kinds of persistent agent context | Dimension | Knowledge (LLM Wiki layer) | Memory (`@coda/memory` layer) | | ----------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------ | | **What it captures** | Domain truths: "Table X joins with Table Y via COUNTRY_CODE" | User behavior: "User A prefers tool X" | | **Source** | Curated sources, schema metadata, expert input | Tool execution traces, query patterns, entity access logs | | **Scope** | Universal per domain — same for all users | Per-user per tenant — specific to one identity | | **Temporal model** | Stable until source changes; revised by linting | Decays without reinforcement; strengthened by repeated behavior | | **Decay behavior** | No decay — a join condition doesn't weaken because no one used it | Exponential decay — preferences weaken without reinforcement | | **Compilation cost** | High (LLM calls per source/lint) | Zero (deterministic aggregation) | | **Compilation trigger** | Explicit (human adds source, requests lint) | Implicit (observations emitted during tool execution) | | **Failure mode** | Wrong knowledge leads to systematically wrong answers | Missing memory leads to unpersonalized but correct answers | | **Criticality** | Load-bearing — agent depends on compiled knowledge for correctness | Best-effort enhancement — agent works fine without it | | **Examples** | Schema relationships, business rules, glossary definitions | Tool preferences, entity frequencies, domain interests, failure patterns | ### Why they belong in separate systems The distinction is not just conceptual — it drives fundamentally different architectural requirements: 1. **Decay semantics are incompatible.** Memory facts _must_ decay — a user's tool preference from three months ago should not persist at full confidence. Knowledge facts _must not_ decay — "Table X joins with Table Y via COUNTRY_CODE" is a structural fact that doesn't weaken over time. Our current `DecayStrategy` interface applies uniformly to all facts. Supporting both requires either conditional decay logic (coupling) or separate fact stores (separation). 2. **Scope semantics conflict.** Memory is per-user (`identityId`-scoped) — different users have different tool preferences and entity access patterns. Knowledge is per-domain — all users querying the royalties domain need the same schema relationships and business rules. Storing both in the same `FactRepository` requires partition logic and complicates the `consolidate()` pipeline. 3. **Ingestion pipelines are incompatible.** Memory observations are emitted automatically during tool execution via fire-and-forget callbacks. Knowledge ingestion requires deliberate source processing — reading schema metadata, extracting relationships, synthesizing summaries. These are different pipelines with different triggers, error handling, and quality requirements. 4. **Prompt injection has different priorities.** Memory facts are ranked by `confidence * reinforcementCount` — the signal strength of learned behavior. Knowledge facts should be ranked by _relevance to the current query_ — a join condition for Table X should appear when the user asks about Table X, not when it was most recently reinforced. The ranking algorithms are different. 5. **Failure modes differ.** Memory failure is benign — the agent works correctly without personalization. Knowledge failure can be actively harmful — stale or incorrect compiled knowledge leads the agent to make wrong recommendations. This warrants different quality assurance: memory can be best-effort; knowledge needs linting, version control, and human review. --- ## Transferable techniques evaluated ### Adopted: None currently The LLM Wiki and `@coda/memory` operate at different layers with different constraints. The techniques below were evaluated for applicability to the memory system specifically. For techniques applicable to the search service, see the [search-layer LLM Wiki comparison](../knowledge/llm-wiki-comparison.md). ### Future applicability: Linting for fact quality **LLM Wiki pattern**: Periodic health checks detect contradictions, stale claims, orphan pages, and coverage gaps. **Assessment**: Our memory system relies exclusively on temporal decay for quality management. A `tool_affinity:get_account` fact and a `tool_avoidance:get_account` fact can coexist if the user's success rate with a tool fluctuates. There is no mechanism to detect contradicting facts within the same identity, facts whose content text has drifted from reality, or categories with suspiciously few facts (suggesting observation gaps rather than genuine user behavior). **Implementation direction**: A lightweight `lintFacts()` function in `@coda/memory` that operates on the already-loaded fact set: | Check | Description | Cost | | ---------------------------- | --------------------------------------------------------------------------------- | ------------------- | | **Contradicting signals** | Same tool appearing in both `tool_affinity` and `tool_avoidance` | O(n) | | **Stale content** | Fact `content` references entities no longer in the search index | Requires search RPC | | **Low-confidence survivors** | Facts near the expiry threshold that keep getting barely reinforced | O(n) | | **Category imbalance** | All facts in one category (e.g., only `tool_affinity`, no entity or domain facts) | O(n) | No LLM calls — pure assertion checks on the fact set. Results could be emitted as log events or surfaced via the future memory visibility API. **Verdict**: Worth pursuing when the memory visibility API is built. Linting facts provides quality assurance that pure decay cannot. The implementation is ~50-100 lines and requires no architectural changes to the memory package. Could optionally lint during consolidation, adding warnings to the `ConsolidationResult`. ### Future applicability: Knowledge feedback loop **LLM Wiki pattern**: Good query answers are filed back into the wiki. Discoveries compound in the knowledge base. **Assessment**: Our memory system has no feedback from agent responses. Observations capture what tools were used and whether they succeeded, but not _what the agent learned from the results_. If the agent discovers that Account 12345 has a complex contract structure requiring specific query patterns, this insight disappears into chat history. **Implementation direction**: This is better addressed at the knowledge layer, not the memory layer. The agent's domain insights are structural facts ("Account 12345 has multi-territory contracts") not behavioral observations ("user accessed Account 12345"). Filing them into `@coda/memory` would require behavioral facts to carry domain semantics they weren't designed for. See the [package architecture section](#package-and-service-architecture) for where this should live. **Verdict**: Valid insight, wrong layer. Knowledge feedback belongs in a compiled knowledge system, not the behavioral memory system. ### Future applicability: Compiled user profiles **LLM Wiki pattern**: Sources are compiled into structured, cross-referenced pages rather than stored as raw data. **Assessment**: Our memory system stores individual facts without synthesis. A user with 30 facts across five categories has no "profile summary" — the facts are injected individually into the prompt and the LLM must synthesize the picture. The LLM Wiki pattern suggests that periodic compilation of individual facts into a higher-order profile could improve prompt efficiency. **Implementation direction**: A `compileProfile()` function that runs after consolidation, synthesizing 30+ individual facts into a 3-5 sentence profile paragraph. Implementation options: 1. **Template-based (no LLM)**: "This user primarily works in the {top_domain} domain, frequently using {top_tools} to access {top_entities}. They tend to encounter issues with {failure_patterns}." Fill from ranked fact categories. Deterministic, zero cost. 2. **LLM-based (higher quality)**: Send the fact set to Claude and ask for a user profile summary. Better prose, but adds LLM dependency and non-determinism to the memory layer. **Verdict**: The template-based approach is worth pursuing as an enhancement to `formatMemorySection()`. It produces a more coherent prompt section than a bullet list of individual facts without breaking the zero-LLM principle. ~30 lines added to the prompt formatter. The LLM-based approach violates our zero-LLM-in-memory principle. ### Rejected: LLM-driven compilation for memory facts **LLM Wiki pattern**: LLM reads sources and synthesizes structured, cross-referenced knowledge pages. **Assessment**: Our memory facts are behavioral statistics derived from counting and averaging. There is nothing for an LLM to synthesize — the consolidation pipeline already produces the optimal representation. "User has used get_account 12 times with 90% success rate, confidence 0.85" cannot be meaningfully improved by LLM processing. The information is already structured. **Verdict**: Rejected. LLM compilation adds value when raw data is unstructured and needs interpretation. Our observations are already structured events with typed metadata. Deterministic aggregation is sufficient. ### Rejected: Wiki-style page structure for memory **LLM Wiki pattern**: Knowledge stored as interlinked markdown pages with `index.md` navigation. **Assessment**: Our memory layer manages ~50 facts per user. A page-per-fact or page-per-category structure would add filesystem overhead without improving retrievability. The full fact set loads in <100ms and fits in ~300 tokens. File-based storage with index navigation is designed for hundreds of pages — far beyond our scale. **Verdict**: Rejected. Redis key-value storage is the right model for our fact volume. Markdown files would add I/O latency and complexity without benefit. ### Rejected: Human-supervised ingestion for memory **LLM Wiki pattern**: Human curates sources, adds them explicitly, guides the LLM's emphasis, reviews output. **Assessment**: Our memory system is fully automatic — observations are emitted during normal tool execution without user involvement. Requiring human curation of memory would defeat the purpose of automatic cross-session learning. Users should not need to "feed" the agent memories; the agent should learn from behavior. **Verdict**: Rejected. Automatic observation is the right model for behavioral memory. Human curation is appropriate for domain knowledge, not user preferences. --- ## Greenfield assessment: if we were building from scratch today ### What we would keep in `@coda/memory` 1. **Observation -> Fact two-stage model.** The separation of raw events (append-only, fire-and-forget) from consolidated beliefs (with confidence and decay) is the right abstraction for behavioral learning. The LLM Wiki's three-layer model solves a different problem (domain knowledge compilation) and should not replace this. 2. **Zero LLM consolidation.** Behavioral memory does not benefit from LLM processing. Counting, averaging, and decaying is the correct consolidation strategy for tool affinity and entity frequency statistics. 3. **Pluggable `DecayStrategy` interface.** Temporal decay is essential for behavioral facts and irrelevant for domain knowledge. This validates that the two concern types need separate handling. 4. **Budget-constrained prompt injection.** The 300-token budget with ranked truncation is the right approach for behavioral context. Domain knowledge injection requires a separate budget and ranking strategy. 5. **Fire-and-forget observation emission.** Behavioral memory writes must not block the critical path. This principle applies only to behavioral observations — domain knowledge ingestion has different latency and quality requirements. 6. **`identityId`-scoped isolation.** Behavioral memory is per-user, never cross-tenant. Domain knowledge would need a different scoping model (per-domain, shared across users within a tenant). ### What we would add 1. **A fact linting pass during consolidation.** Detect contradicting signals, stale content, and category imbalance. Zero LLM cost — pure assertion checks on the fact set. Log warnings; surface via the future memory visibility API. 2. **Template-based profile compilation.** Synthesize 30+ individual facts into a 3-5 sentence profile paragraph for more coherent prompt injection. Deterministic, ~30 lines in the prompt formatter. 3. **A clear architectural boundary between memory and knowledge.** The memory package should own behavioral facts (per-user, decaying, automatically observed). Domain knowledge (per-domain, stable, compiled from sources) should live in a separate system. This prevents semantic confusion (should join conditions decay?), engineering confusion (should knowledge use the same decay strategy?), and scope confusion (should knowledge be per-user or shared?). ### What we would not adopt from the LLM Wiki for memory 1. **LLM-driven compilation.** Behavioral facts don't benefit from LLM synthesis. The data is already structured. 2. **Wiki page storage.** ~50 facts per user don't warrant filesystem navigation. 3. **Human-supervised ingestion.** Memory must be automatic. 4. **Shared scope.** Behavioral memory must remain per-user. ### What we would adopt from the LLM Wiki as a separate system 1. **Compiled domain knowledge.** The LLM Wiki's core thesis — "compile once, keep current" — applies directly to the gap between raw schema metadata (what the search service returns) and domain understanding (what the agent needs to interpret results). This is not memory; it is a separate knowledge layer. 2. **Proactive quality maintenance.** The linting pattern — detecting contradictions, staleness, orphans, and coverage gaps — applies to both behavioral facts (lightweight) and domain knowledge (heavier). 3. **Knowledge feedback loop.** The pattern of filing discoveries back into a persistent store applies to domain knowledge, not behavioral memory. When the agent discovers a schema relationship or a business rule, that discovery should compound — but in the knowledge layer, not the memory layer. --- ## Package and service architecture The analysis above identifies two fundamentally different concern types that should not be conflated. This section evaluates where each should live in Coda's package and service structure. ### Current state ``` @coda/memory Behavioral learning: observations -> facts -> prompt injection @coda/extensions Static domain glossary: hand-curated JSON files loaded at startup @coda/search Search infrastructure: BM25 + HNSW + glossary matching + RRF fusion apps/search Search service: embedding, indexing, schema polling apps/server Agent server: memory service, tool execution, agent loop ``` The glossary in `@coda/extensions` is already a rudimentary compiled knowledge layer — static JSON files mapping domain terms to schema targets with context strings. But it is hand-curated, rarely updated, and cannot grow from agent interactions or automated analysis. ### Where compiled knowledge should live There are three architectural options for a compiled knowledge layer: #### Option A: Extend `@coda/extensions` with richer glossary entries | Aspect | Assessment | | ------------------ | --------------------------------------------------------------------------------------------------------------------- | | **What changes** | Add `relationships`, `gotchas`, `related_concepts` fields to `GlossaryEntry`. Add new glossary JSON files per domain. | | **Pros** | Zero new infrastructure. Follows existing pattern. Both services already import `@coda/extensions`. | | **Cons** | Still static — no dynamic compilation, no feedback loop, no linting. Manual curation only. | | **When to choose** | When the knowledge is stable enough to maintain by hand (few dozen entries, infrequent changes). | This is the lowest-cost option and was already recommended in the [search-layer LLM Wiki comparison](../knowledge/llm-wiki-comparison.md) as "Idea 1: Glossary enrichment." It addresses knowledge _richness_ but not knowledge _accumulation_ or _maintenance_. #### Option B: New `@coda/knowledge` package (recommended) | Aspect | Assessment | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **What changes** | New package with distinct data model, ingestion pipeline, and prompt injection. | | **Pros** | Clean separation from behavioral memory. Can model knowledge facts without decay. Can be shared across users. Can grow from automated analysis or LLM compilation. | | **Cons** | New package to maintain. Needs its own storage strategy. | | **When to choose** | When knowledge needs to grow dynamically, be compiled from sources, or be shared across users. | **Proposed `@coda/knowledge` design:** ``` packages/knowledge/ src/ types.ts KnowledgeFact, KnowledgeSource, KnowledgeEntry knowledge-store.ts Interface for CRUD on knowledge facts (no decay) compiler.ts Source -> compiled knowledge (template or LLM) lint.ts Health checks: contradictions, staleness, coverage prompt-formatter.ts "What you know about this domain" section index.ts Public API docs/ architecture.md ``` **Key design differences from `@coda/memory`:** | Concern | `@coda/memory` | `@coda/knowledge` (proposed) | | --------------------- | -------------------------------------------- | ------------------------------------------------------------ | | **Fact type** | `Fact` with confidence + decay | `KnowledgeFact` with source + authority + version | | **Scope** | Per-`identityId` | Per-domain or per-tenant (shared across users) | | **Decay** | Exponential or adaptive | None — facts persist until source changes | | **Ingestion** | Automatic from tool execution | Explicit from schema analysis, admin API, or LLM compilation | | **Consolidation** | Deterministic aggregation | Optional LLM synthesis (or template-based) | | **Quality assurance** | Temporal decay | Linting: contradictions, staleness, coverage gaps | | **Prompt injection** | "What you know about this user" (300 tokens) | "What you know about this domain" (separate budget) | | **Storage** | Redis hash maps | Aurora MySQL (queryable for compliance/analytics) | | **Version control** | None (facts overwritten) | Source-linked with version history | **Integration in `apps/server`:** ``` apps/server/src/ai/ memory/ Wraps @coda/memory (behavioral, per-user) memory-service.ts observation-emitters.ts prompt-formatter.ts affinity.ts knowledge/ Wraps @coda/knowledge (domain, shared) <- NEW knowledge-service.ts prompt-formatter.ts ``` `MemoryService.loadMemory()` and `KnowledgeService.loadKnowledge()` both run at session start. Their prompt sections are injected separately: ``` System prompt = knowledge.promptSection <- "What you know about this domain" + memory.promptSection <- "What you know about this user" + langfusePrompt <- Domain instructions ``` #### Option C: New `apps/knowledge` service | Aspect | Assessment | | ------------------ | -------------------------------------------------------------------------------------------------------------- | | **What changes** | Separate microservice for knowledge compilation, storage, and retrieval. | | **Pros** | Full isolation. Can run LLM compilation without impacting agent latency. Can scale independently. | | **Cons** | New infrastructure (Fargate, networking, monitoring). RPC latency for knowledge loading. Operational overhead. | | **When to choose** | When knowledge compilation requires heavy LLM processing that would impact agent server performance. | **Assessment**: A separate service is premature. Knowledge loading at session start can use the same pattern as memory loading — race against a timeout, return empty on failure. The volume (~100-500 knowledge facts per domain) does not warrant dedicated infrastructure. LLM-based compilation, if adopted, can run as a background job in `apps/server` or `apps/search`. **Verdict**: Start with Option B (`@coda/knowledge` package) and revisit Option C if compilation cost or knowledge volume outgrows in-process handling. ### Why not extend `@coda/memory` directly? Extending the memory package to handle both behavioral facts and domain knowledge would require: 1. **Conditional decay logic** — skip decay for knowledge-category facts in `consolidate()`. This couples two different temporal models in one pipeline. 2. **Dual scope** — `identityId` for behavioral facts, domain/tenant scope for knowledge facts. The `FactRepository` interface assumes identity-scoped storage. 3. **Dual prompt sections** — `formatMemorySection()` currently produces one section. Mixing "User prefers get_account" with "Table X joins Table Y via COUNTRY_CODE" under one heading confuses the agent. 4. **Dual ingestion pipelines** — observation emitters (fire-and-forget from tool execution) and knowledge compilers (deliberate, possibly LLM-assisted) have fundamentally different quality and latency requirements. 5. **Dual ranking strategies** — behavioral facts ranked by `confidence * reinforcementCount`; knowledge facts ranked by query relevance. Each of these requires special-casing in the consolidation pipeline, fact repository, and prompt formatter. The result is a package that does two things poorly rather than one thing well. A separate `@coda/knowledge` package avoids this coupling. --- ## Relationship to prior comparisons | System | Layer | Relationship to Coda's memory | | -------------------------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------- | | [Stash](stash-comparison.md) | Memory | Agent memory layer — inspired `@coda/memory` design (observation -> fact consolidation, decay) | | [MemOS](memos-comparison.md) | Memory OS | Full memory operating system — validates design choices, identifies future gaps | | [Memori](memori-comparison.md) | Memory | Production memory infrastructure — highlights execution trace capture and SQL storage gaps | | **LLM Wiki (memory)** | Knowledge | Pre-compiled knowledge pattern — reveals that domain knowledge and behavioral memory are distinct concerns requiring separate systems | | [LLM Wiki (search)](../knowledge/llm-wiki-comparison.md) | Knowledge | Same pattern applied to search — identifies glossary enrichment and entity summary opportunities | The four memory-layer comparisons (Stash, MemOS, Memori, LLM Wiki) have converged on a consistent architectural picture: **Stash** inspired the observation-to-fact pipeline. **MemOS** validated the two-stage model while surfacing gaps in procedural memory and user visibility. **Memori** highlighted richer execution trace capture and SQL-native storage needs. Each of these three compared systems in the same layer as `@coda/memory` — agent memory — and found our architecture sound with incremental improvements needed. **LLM Wiki** reveals the most fundamental finding by operating at a _different_ layer: **behavioral memory and domain knowledge are separate concerns with incompatible temporal models, scope semantics, and quality requirements**. The prior three comparisons evaluated systems in the same layer as `@coda/memory` (agent memory) and found our architecture sound with incremental improvements needed. The LLM Wiki comparison evaluates a system in a _different layer_ (compiled knowledge) and finds that a new architectural component is warranted — not as a replacement for memory, but as a complement. ### The emerging stack ``` Domain Knowledge Layer (NEW — @coda/knowledge) Compiled, stable, shared across users, no decay Source: schema analysis, expert input, LLM compilation Prompt: "What you know about this domain" Quality: linting (contradictions, staleness, coverage) Behavioral Memory Layer (@coda/memory) Learned, temporal, per-user, confidence decay Source: tool execution observations Prompt: "What you know about this user" Quality: decay + reinforcement Schema Retrieval Layer (@coda/search + apps/search) Indexed, real-time, multi-signal hybrid search Source: live schema polling (GraphQL, Snowflake) Returns: ranked schema items for agent consumption Quality: BM25 + HNSW + glossary + graph + RRF Static Glossary Layer (@coda/extensions) Curated, stable, loaded at startup Source: hand-authored JSON glossary files Consumed by: search (embedding enrichment + query boost) Quality: manual review (future: linting) ``` The knowledge layer sits between the static glossary (too simple to grow) and the behavioral memory (too per-user to share). It is the LLM Wiki's contribution: a compiled, maintained, shared knowledge store that enriches the agent's understanding of the domain. --- ## Project maturity assessment | Factor | LLM Wiki | Coda `@coda/memory` | | ------------------ | ----------------------------------------------------------------------------- | ---------------------------------------------------------------- | | **Age** | Published April 2026; architectural pattern, not a project | Implemented 2026; ~3 months in production | | **Maintainership** | Solo author (Andrej Karpathy); community discussion in gist comments | Solo team; embedded in monorepo | | **Implementation** | None — intentionally abstract pattern, no reference code | ~900 LoC TypeScript with comprehensive test suite | | **Documentation** | Single gist (~2,500 words) describing the pattern | Architecture doc + design exploration + inline JSDoc | | **Adoption** | Widely discussed; adopted by individual practitioners with various LLM agents | Internal production deployment | | **Dependencies** | Any LLM, any file system, optional search tooling | Redis (in-memory fallback); zero external dependencies | | **Maturity model** | Idea-stage: proven in personal use, no production validation at scale | Production: deployed with graceful degradation and test coverage | The LLM Wiki is a design pattern, not a software project. It cannot be adopted as a dependency — it must be implemented from scratch, adapted to the specific domain and constraints. Coda's memory system is a production feature with clear interfaces, test coverage, and graceful degradation. The comparison is between an architectural idea and a running system. --- ## Summary The LLM Wiki pattern and Coda's `@coda/memory` solve related but fundamentally different problems: - **LLM Wiki**: "The agent keeps re-deriving domain understanding from raw data. Compile knowledge once, keep it current, and serve the compiled result." - **`@coda/memory`**: "The agent forgets what it learned about the user between sessions. Accumulate observations, consolidate into facts, and inject personalized context." Both are about avoiding re-derivation, but they operate at different layers with incompatible requirements. Domain knowledge is stable, shared, and compiled from sources. Behavioral memory is temporal, per-user, and learned from observation. The two should not be conflated in a single system because their temporal models (decay vs. persistence), scope semantics (per-user vs. per-domain), ingestion pipelines (automatic vs. deliberate), and failure modes (benign vs. harmful) are fundamentally different. The LLM Wiki's most valuable contributions to Coda are: 1. **The architectural insight**: behavioral memory and domain knowledge are separate concerns that warrant separate systems. Mixing them in `@coda/memory` would compromise both. 2. **The linting pattern**: proactive quality maintenance (contradictions, staleness, coverage gaps) is superior to relying solely on temporal decay. A lightweight fact linting pass can be added to `@coda/memory` with zero LLM cost. 3. **The feedback loop**: knowledge should compound from agent interactions. This applies to domain knowledge (filing schema discoveries as persistent facts), not behavioral memory (which already compounds via observation reinforcement). **Bottom line**: Create a new `@coda/knowledge` package to house compiled domain knowledge — schema relationships, business rules, and synthesized entity context — separate from the behavioral memory in `@coda/memory`. The two systems share a session-start loading pattern and prompt injection mechanism but differ in data model, temporal semantics, ingestion pipeline, and scope. The existing `@coda/extensions` glossary continues as the static input layer; `@coda/knowledge` is the dynamic compiled layer between glossary and memory. Start with template-based compilation (zero LLM cost), add LLM-based compilation as an option when the value is demonstrated, and implement linting in both the memory and knowledge layers. --- ## References - Karpathy, A.: [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) (2026) — architectural pattern for persistent LLM knowledge bases - LLM Wiki search comparison: [llm-wiki-comparison.md](../knowledge/llm-wiki-comparison.md) — prior analysis against the search service - MemOS comparison: [memos-comparison.md](memos-comparison.md) — memory OS evaluation - Memori comparison: [memori-comparison.md](memori-comparison.md) — agent memory infrastructure evaluation - Stash comparison: [stash-comparison.md](stash-comparison.md) — agent memory layer evaluation - Agent memory design exploration: [agent-memory-exploration.md](../../archive/agent-memory-exploration.md) - Agent memory architecture: [agent-memory.md](../../../architecture/agent-memory.md) - Memory package: `packages/memory/src/` - Memory service integration: `apps/server/src/ai/memory/` - Extensions package (glossary): `packages/extensions/src/`