> **Pre-implementation design exploration.** This document was written before @coda/memory was implemented. See [Agent Memory Architecture](../../architecture/agent-memory.md) for the actual implemented design.

# Agent Memory — Design Exploration

## Context

This document explores adding cross-session memory to the Coda agent, inspired by patterns from [Stash](https://github.com/alash3al/stash) (see [comparison](../comparisons/memory/stash-comparison.md)). It is a design exploration, not a spec — it identifies opportunities, proposes architecture, and surfaces trade-offs for discussion.

---

## Problem statement

The Coda agent is stateless across sessions. Every conversation starts with the same static system prompt, the same tool catalog, and zero knowledge of the user's history with the platform. Specifically:

1. **No user context**: The agent doesn't know that a user primarily works with royalty data, prefers Snowflake over GraphQL, or always asks about the same three tables. Every session re-discovers this from scratch.

2. **No failure memory**: If `query_snowflake` times out on a particular query pattern, the agent learns this within the conversation but forgets by the next session. The same failure repeats across sessions with no adaptation.

3. **No goal continuity**: A user building a dashboard over three conversations repeats context each time. The agent cannot resume from where it left off — it has no concept of multi-session tasks.

4. **No tool affinity**: Tool selection is driven by the static system prompt and glossary boost scoring (`apps/server/src/ai/tools/catalog.ts`). A user who has never used GraphQL tools still sees them ranked equally. The agent cannot learn which tools are productive for which users.

### What exists today

| Mechanism                        | Scope            | TTL       | Cross-session?                                        |
| -------------------------------- | ---------------- | --------- | ----------------------------------------------------- |
| Redis conversation cache         | Per conversation | 7 days    | No — each conversation is isolated                    |
| Conversation history (20 msgs)   | Per conversation | Session   | No — capped and not shared                            |
| System prompt                    | Global           | Static    | N/A — identical for all users                         |
| Tool glossary                    | Global           | Static    | N/A — no per-user adaptation                          |
| DB persistence (StreamPersister) | Per conversation | Permanent | Theoretically — but not read back into the agent loop |

The agent has durable storage (Aurora via StreamPersister) but never reads it back. Conversations are persisted for audit/display, not for agent learning.

---

## Proposed architecture

### Design principles

1. **No LLM calls in the memory layer** — consolidation must be deterministic and cheap. This rules out Stash's LLM-dependent fact extraction but not the pattern itself.
2. **Per-tenant isolation** — memory is scoped to `identityId` (the same key used for conversation storage). No cross-tenant leakage.
3. **Additive to the system prompt** — memory surfaces as additional context in the system prompt, not as a separate retrieval mechanism. The LLM decides how to use it.
4. **Graceful degradation** — if the memory store is unavailable, the agent falls back to the current stateless behavior. Memory is an enhancement, not a dependency.
5. **Budget-constrained** — memory injected into the system prompt has a fixed token budget. The agent's context window is not unbounded.

### Data model

Three entity types, modeled after Stash's progression but without LLM-dependent stages:

```
┌─────────────┐     consolidation      ┌─────────────┐
│ Observation  │ ──────────────────────▶│    Fact      │
│ (append-only)│   (BM25 clustering)   │ (with decay) │
└─────────────┘                        └──────┬──────┘
                                              │
                                       read into system prompt
                                              │
                                              ▼
                                     ┌─────────────────┐
                                     │  Agent session   │
                                     └─────────────────┘
```

#### Observation (raw event, append-only)

Recorded automatically after each conversation turn. Not read by the agent directly — serves as input to consolidation.

```typescript
interface Observation {
  id: string;
  identityId: string;
  type: ObservationType;
  content: string;
  metadata: Record<string, unknown>;
  createdAt: Date;
}

type ObservationType =
  | "tool_success" // Tool X returned useful results for query Y
  | "tool_failure" // Tool X failed with error Y
  | "entity_access" // User accessed entity X (table, type, product)
  | "query_pattern" // User asked about domain X with terms [Y, Z]
  | "session_summary"; // End-of-session summary (auto-generated from conversation)
```

Source: the `onToolExecution` callback in `ConverseWithToolsOptions` already fires after each tool round with tool uses, results, and timing. Observations can be emitted here with zero changes to the orchestrator loop.

#### Fact (consolidated belief, with confidence decay)

Derived from observations by deterministic aggregation. Read into the system prompt at session start.

```typescript
interface Fact {
  id: string;
  identityId: string;
  category: FactCategory;
  content: string; // Human-readable summary
  confidence: number; // 0.0–1.0, decays without reinforcement
  reinforcementCount: number; // How many observations support this
  lastReinforcedAt: Date;
  createdAt: Date;
  updatedAt: Date;
  expiresAt: Date | null; // Soft expiry when confidence drops below threshold
}

type FactCategory =
  | "domain_preference" // "User primarily works with royalty data"
  | "tool_affinity" // "User frequently uses query_snowflake successfully"
  | "tool_avoidance" // "query_graphql consistently fails for this user"
  | "entity_frequency" // "User accesses CONTRACT and STATEMENT_PERIOD tables often"
  | "failure_pattern" // "Snowflake queries with >5 JOINs tend to timeout"
  | "workflow_pattern"; // "User typically searches → queries → exports"
```

#### Goal (optional, future — multi-session task tracking)

Deferred to a later phase. Requires the agent to explicitly create/update goals, which adds tool-use complexity. The observation/fact model provides value without this.

### Consolidation (no LLM)

Consolidation runs as a background job (cron or post-session hook). It processes new observations since the last checkpoint and updates facts.

**Algorithm:**

1. Fetch observations since `lastCheckpointId` for the tenant
2. Group by type (tool_success, entity_access, query_pattern, etc.)
3. For each group:
   - **tool_success/tool_failure**: Increment/decrement tool affinity facts. Key: `{identityId}:{toolName}`. If a fact exists, reinforce (bump confidence + count). If not, create with initial confidence 0.5.
   - **entity_access**: Increment entity frequency facts. Key: `{identityId}:{entityFQN}`. Use BM25 tokenization from `@coda/search` to cluster similar entity names.
   - **query_pattern**: Extract domain terms via tokenization. Update domain preference facts. Key: `{identityId}:{domain}`.
   - **session_summary**: Not consolidated — stored for auditability only.
4. **Confidence decay**: For all facts not reinforced in the current run, multiply confidence by decay factor (e.g., 0.95 per day). Expire facts below threshold (e.g., 0.1).
5. Update checkpoint.

This is pure aggregation — counting, averaging, and decaying. No LLM calls, no embedding, no vector search. The consolidation logic is ~200 lines of TypeScript using existing `@coda/search` tokenization.

### System prompt injection

At session start, load the tenant's active facts (confidence > threshold, not expired) and inject them as a memory section in the system prompt:

```markdown
## Your memory of this user

Based on previous conversations with this user:

- They primarily work with **royalty accounting** data (Snowflake)
- Their most-accessed tables: STATEMENT_PERIOD, CONTRACT, ROYALTY_LINE_ITEM
- query_snowflake succeeds 92% of the time; query_graphql succeeds 45%
- Queries with >5 JOINs have timed out 3 times — suggest breaking into steps
- Common workflow: search for tables → preview data → build aggregation query

Use this context to prioritize tools and tailor responses. This memory updates
over time — if it seems outdated, the user's needs may have changed.
```

**Token budget**: ~300 tokens max. Facts are ranked by `confidence * reinforcementCount` and truncated to budget. Top-K facts only — this is a prompt enhancement, not a knowledge dump.

### Tool affinity bias

The tool catalog (`searchCatalog` in `apps/server/src/ai/tools/catalog.ts`) already uses glossary boosting. Tool affinity facts can provide a per-user boost:

```typescript
// In searchCatalog(), after glossary boost:
const affinityBoost = toolAffinityFacts.get(entry.name) ?? 0;
if (affinityBoost > 0) score += affinityBoost * 2;
```

This biases tool discovery toward tools the user has successfully used before. The boost is proportional to confidence (which decays), so it naturally adapts as usage patterns change.

### Storage options

| Option                                        | Pros                                                                                      | Cons                                                              |
| --------------------------------------------- | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| **Redis (hashes)**                            | Already in stack, low latency, natural TTL for decay                                      | No query flexibility, limited to key-value patterns               |
| **Aurora (new table)**                        | Queryable, durable, consistent with StreamPersister                                       | Cold start latency, requires migration, overkill for simple facts |
| **Redis for observations + Aurora for facts** | Observations are high-volume/ephemeral (Redis TTL); facts are low-volume/durable (Aurora) | Two stores to maintain                                            |

**Recommendation**: Redis for both, initially. Observations are append-only with a 30-day TTL. Facts are small (dozens per tenant, not thousands). Redis hashes with `identityId` as key provide O(1) lookup at session start. If we later need queryability across tenants (e.g., "which tools fail most often globally"), we can add Aurora as a secondary store.

---

## Integration points

### Where observations are emitted

| Source         | Integration point                                        | Observation type                 |
| -------------- | -------------------------------------------------------- | -------------------------------- |
| Tool execution | `onToolExecution` callback in `ConverseWithToolsOptions` | `tool_success` / `tool_failure`  |
| Search results | `SearchSnowflake` / `SearchGraphQL` tool handlers        | `entity_access`, `query_pattern` |
| Session end    | `stream-handler.ts` post-conversation hook               | `session_summary` (optional)     |

The `onToolExecution` callback already receives `toolUses`, `toolResults`, and timing for every tool round. Emitting observations requires ~10 lines per integration point — constructing the `Observation` object and calling `observationStore.append()`.

### Where facts are consumed

| Consumer        | Integration point                                                              | Fact categories used                    |
| --------------- | ------------------------------------------------------------------------------ | --------------------------------------- |
| System prompt   | `converseWithTools()` in `orchestrator.ts` — prepend to system prompt          | All                                     |
| Tool catalog    | `searchCatalog()` in `catalog.ts` — per-user boost                             | `tool_affinity`, `tool_avoidance`       |
| Thinking budget | `thinking-budgets.ts` — adjust budget based on user's typical query complexity | `domain_preference`, `workflow_pattern` |

### What doesn't change

- The orchestrator loop (`converseWithTools`) — no changes to the tool-use loop itself
- The conversation store — per-conversation history is unchanged
- The search service — memory is in the agent layer, not the retrieval layer
- Tool handlers — they return results as before; observation emission is in the callback layer

---

## Confidence decay model

Stash's decay model applies here because user preferences genuinely change over time (unlike schema metadata, which is authoritative). The model:

```
confidence_new = confidence_old × decay_factor    (per decay interval)
```

| Parameter           | Value                   | Rationale                                                                                            |
| ------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------- |
| Decay factor        | 0.95                    | 5% reduction per interval — slow enough to retain strong patterns, fast enough to age out stale ones |
| Decay interval      | 1 day                   | Daily cron job. Agent usage is daily for active users.                                               |
| Expiry threshold    | 0.1                     | Below 10% confidence, the fact is no longer worth injecting into the prompt                          |
| Reinforcement boost | min(1.0, current + 0.1) | Each re-observation adds 0.1 confidence, capped at 1.0                                               |

A fact reinforced daily stays near 1.0 indefinitely. A fact not reinforced for 30 days decays to ~0.21. After 45 days without reinforcement, it expires (~0.10). This naturally ages out stale patterns while preserving durable ones.

---

## What this does NOT cover

1. **Multi-session goal tracking** — deferred. Requires the agent to explicitly manage goals (create, update, complete), which adds tool-use complexity. The observation/fact model provides value without this.
2. **Cross-tenant learning** — explicitly excluded. No "users who searched for X also searched for Y" patterns. Memory is per-tenant only.
3. **Glossary evolution** — the glossary (`packages/extensions/`) could benefit from query pattern observations, but this is a separate initiative. BM25 term frequency analysis on `query_pattern` observations could identify glossary gaps without LLM calls.
4. **Agent self-improvement** — the agent doesn't modify its own system prompt or tool definitions. Memory is read-only context, not self-modifying behavior.
5. **Stash integration** — we are adopting patterns, not the tool. Stash's PostgreSQL + LLM consolidation architecture doesn't fit our stack.

---

## Effort estimate

| Component                 | Scope                                                              | Complexity       |
| ------------------------- | ------------------------------------------------------------------ | ---------------- |
| Observation store (Redis) | New interface + Redis implementation                               | ~150 lines       |
| Fact store (Redis)        | New interface + Redis implementation                               | ~150 lines       |
| Consolidation job         | Background worker, BM25 clustering, decay                          | ~300 lines       |
| Observation emitters      | 3 integration points (tool callback, search handlers, session end) | ~50 lines        |
| System prompt injection   | Load facts, format, prepend to prompt                              | ~80 lines        |
| Tool affinity boost       | Modify `searchCatalog()` scoring                                   | ~20 lines        |
| Tests                     | Unit + integration for stores, consolidation, injection            | ~400 lines       |
| **Total**                 |                                                                    | **~1,150 lines** |

No new infrastructure — Redis is already in the stack. No new dependencies. No LLM calls. No schema migrations (Redis-only initially).

---

## Open questions

1. **Privacy**: Should users be able to see/edit/delete their memory? If yes, this needs an API endpoint and client UI. If no, we need to be transparent about what's stored.
2. **Opt-out**: Should memory be opt-in or opt-out? Enterprise tenants may have data retention policies that conflict with accumulating user behavior data.
3. **Cold start**: New users have no facts. The system prompt memory section is empty. Is this fine (graceful degradation), or should we seed with tenant-level defaults?
4. **Session summary quality**: Without LLM calls, session summaries would be mechanical (tool names + entity names accessed). Is this useful enough, or should we skip session summaries entirely and rely on granular observations?
5. **Observation volume**: Active users might generate 50–100 observations per day. At 30-day TTL, that's ~3,000 observations per tenant in Redis. Is this acceptable, or should we consolidate more aggressively?

---

## References

- Stash comparison: [stash-comparison.md](../comparisons/memory/stash-comparison.md)
- Agent orchestrator: `apps/server/src/ai/orchestrator.ts`
- Tool catalog: `apps/server/src/ai/tools/catalog.ts`
- Conversation store interface: `apps/server/src/cache/conversation-store.ts`
- Thinking budgets: `apps/server/src/ai/thinking-budgets.ts`
- Stream handler: `apps/server/src/routes/stream-handler.ts`