Architecture
Building memory an agent can trust
We saved a rule, then watched the agent ignore it. The lesson was simple: keeping a reliable record and building useful context are separate jobs. Here is how the design evolved across file-backed and database-backed implementations.
1. Start with a few giant markdown files
We began with readable files loaded into every turn. As they grew, each request carried more irrelevant context, cost and stale material. A loading limit could cut the tail entirely. Then a concurrent-write test exposed a second problem: both saves succeeded, but one rule disappeared.
- Recurring inputCaching can reduce price; it does not remove the context footprint.
- Context rotUseful instructions compete with stale, duplicate and unrelated text.
- Lost tailA hard cut means some text never reaches the model.
How two successful saves lost one rule
[R0]
[R0]
[R0, A]
[R0, B]
Final file: [R0, B] · A is lost, although both writers reported success.
2. Separate organization and user context
We moved to one record per claim, with an owner, source and history. Writing chooses where a claim belongs. Reading chooses which records the current actor may see. Sharing useful knowledge and preserving private context need different gates.
In the checked implementation, the org-write gate used model-reported evidence and a heuristic affiliation check that allowed unknown cases. It was not independent verification. User rules had a standing block; org memories still relied on search.
Reference record schema and the write/read checks
MemoryRecord {
id, kind, body
scope: { organization, user?, project?, tool?, agent? }
source: { conversation?, message?, evidence? }
valid_from, valid_until?
supersedes?, checked_at?
}
The schema is conceptual. The actual organization lesson gate accepted user-stated or verified-in-data labels with an affiliation check; otherwise the lesson stayed with its author. Organization filtering and actor-visible references constrained reads. The inspected 9,380 recalled items had no cross-user or cross-organization leaks; that is evidence for the sample, not a universal guarantee.
3. Give memory types different read policies
A stored “no sales tax” instruction was missed when the next request simply asked for an invoice. We separated applicable instructions from task-relevant search. Facts, episodes and procedures can compete for relevance; standing rules should not depend on the query repeating their words.
Explore three requests through the two lanes
A record exists: “Do not charge sales tax on our invoices.”
Include the invoice instruction even though the request does not mention tax.
Find invoice context. Similarity search alone can miss the tax rule.
The reported rules block still overflowed: one user saw 23 of 135 rules. Short replies could skip it too. Separating the lanes is the design requirement; those defects were not established as resolved.
Memory types and the reference read algorithm
| Kind | Example | How it should be used |
|---|---|---|
| Standing instruction | “Do not charge sales tax on our invoices.” | Consider whenever its scope applies, even if search would miss it. |
| Fact | “The warehouse closes at 5.” | Retrieve when relevant; recheck if it can change. |
| Episode | “Last week's invoice reconciliation.” | Help locate earlier work and its evidence. |
| Procedure | “How to prepare the daily reconciliation.” | Retrieve as a coherent workflow, with its preconditions. |
build_memory(actor, task, budget):
eligible = records_visible_to(actor)
rules = applicable_instructions(eligible, actor, task)
candidates = search(eligible, task)
ranked = rerank(candidates, task)
block = pack(rules, ranked, budget)
return block, selection_trace
Authorization precedes relevance. The packer should expose what did not fit; different implementations use different selectors.
4. Optimize reads: rank for the task
A smaller prompt was still noisy. TypeSafe’s Jev introduction gave us a structured scorer to explore. We added task-aware reranking after broad candidate search, with word ordering as a fallback. Later, a separate session-start path kept injecting unrelated tunnel notes. Improving one entrance had not fixed the others.
Candidate snippets went to an external scoring service, adding privacy and latency tradeoffs. A live session’s tunnel lines fell from eight to zero after a scoping fix; fleet-wide task gains were not proven.
5. Optimize reads: delivery and cache placement
Good selection did not guarantee delivery. The harness exposed a preview of a much larger memory block, and the overflow was never opened. We bounded the block and started checking the model-visible prompt rather than trusting the hook output.
PREVIEW CUT
45 of 54 inspected injections were cut.
Preserve the reusable prefix.
Put changing memory in the adapter’s appropriate position.
An 8,000-character cap addressed that integration. It is not a portable budget. A different harness needs its own visibility and cache checks.
6. Maintain records without losing meaning
Revisiting the design through Anthropic’s memory-components article sharpened the maintenance question. A repeat summary silently archived instructions it omitted: 74 user-stated rules were lost in two weeks. We stopped treating omission as withdrawal. Explicit replacements retained the old version, but near-duplicate merging could still remove an obligation.
Use ALL CAPS for category codes and size entries.
Use ALL CAPS for applicable category codes.
The replacement no longer carries the size-entry obligation.
- Find a candidateDuplicates, contradictions and changing environment facts.
- Follow the sourceA similar sentence is not proof of the same obligation.
- Change the active setExplicit replacement or retirement; preserve history and leave unresolved claims reviewable.
Cleanup results and the remaining merge risk
The restore repair recovered 76 records, a different set from the 74-rule loss sample. A later write replay turned 28 of 316 saves into updates; 10 of 12 sampled merges were judged right. One-time cleanup retired 22 facts across 12 groups. The two questionable merges are why similarity alone cannot authorize superseding a rule.
7. Optimize maintenance with measured feedback
The “used” field only meant selected for a brief. We needed separate evidence at each boundary. A 26-case experiment supported preserving lessons and supplying standing rules, but its improvement interval crossed zero. It was direction for the next iteration, not proof that memory was solved.
−0.01 to +0.24 · crosses zero
26 cases · two automated judges · no human validation- Written?The right claim and scope survived the save.
- Selected?The needed memory was picked for the task.
- Delivered?The full text reached the model.
- Applied?The answer followed the rule correctly.
- Outcome improved?Less rework or better success, latency and cost.
Coverage counts and evaluation method
| Memory supplied | Rule-following score | Needed rule reached the model |
|---|---|---|
| None | 0.51 | 0 of 26 |
| Existing implementation | 0.72 | 11 of 26 |
| Preserve lessons across summaries | 0.77 | 14 of 26 |
| Preserve lessons + standing rules | 0.83 | 24 of 26 |
Three answers per case, two blind automated judges, 78% judge agreement and no human validation. The larger planned run had not been performed. Next checks should include simultaneous saves, omitted rules, short replies, overflow, meaning-changing merges and slow search.
The final design: a scoped memory lifecycle
The store remembers what was learned. The read path chooses what belongs in this turn. The maintenance path changes what is still considered true. Each path returns a trace, so a lost instruction can be located at the boundary that failed.
Control the mutation
- Classify kind and scope
- Check authority and evidence
- Compare related claims
- Commit a version + event
Keep the claim and its history
- Identity
- ID + organization + scope owner
- Meaning
- Kind + full claim
- Provenance
- Source + evidence + checked date
- Lifecycle
- Version + active dates + supersedes
Separate search index
Append-only change events
Build this turn’s context
- Filter by actor visibility
- Load applicable rules
- Search + rank context
- Pack with exclusion reasons
- Adapt to the harness
Find drift → verify evidence → keep / replace / retire / review.
Join record versions to actual prompts, behavior and task outcomes.
Follow a record through the design
Write a claim without widening its authority
- ProposeExtract a durable claim; keep its original source reference.
- RouteChoose its kind and destination: organization, user, tool or agent.
- GateCheck permission to write that scope and the evidence policy. Similarity never grants broader authority.
- CompareFind related active records in the same scope. Decide add, explicit replacement, or review.
- CommitSave the new version, supersession link and event together; retire the old version only after the replacement exists.
Reference requirement: unknown authority must not silently promote a personal claim to organization memory. This is stronger than the permissive affiliation heuristic found in the checked code.
Construct the memory for this turn
- ConstrainResolve organization and actor-visible records before searching. Another user’s private records cannot enter the candidate pool.
- Load rulesSelect applicable standing instructions independently of query length and search availability.
- RetrieveSearch the remaining eligible facts, episodes and procedures; rerank against the task, with a defined fallback.
- PackDeduplicate and allocate budgets by kind. Return the chosen versions and the reasons other records did not fit.
- DeliverPlace changing memory around the harness’s cache boundary. Verify the actual text the model receives, not only the service response.
Reference requirement: a search timeout may reduce retrieved context but must not silently remove standing instructions. Overflow must be visible; a capped list is not proof all rules arrived.
Keep the active set accurate without erasing history
- FindIdentify stale candidates, repeated subjects and conflicting claims; use similarity to nominate pairs only.
- VerifyOpen the supporting source. Separate a changed fact from an instruction that only its authorized owner can change.
- DecideKeep, explicitly supersede, retire a disproven fact, or leave the claim unresolved for review.
- ApplyUse the same controlled write path. Preserve previous versions, evidence and the reason for the change.
- MeasureCheck that the update preserved obligations and improved later reads or outcomes before treating cleanup as a success.
Reference requirement: omission by a later summary is never withdrawal. If the source cannot settle a conflict, the system must expose uncertainty rather than guess a replacement.
Where the implementation record stops
- Built or observedScoped records, history, an org/user write gate, standing user rules, bounded recall, reranking and a one-time cleanup.
- Requirements still exposed by failuresSafe unknown-authority handling, complete rule delivery under overflow, short-reply independence and meaning-preserving replacements.
- Not establishedOperational nightly source verification and a controlled improvement in real task outcomes.
Readings behind the design
Articles and projects we explored
- Dhravya Shah: Reverse-engineering Instinct’s memory — compact profiles, on-demand reads and background record maintenance. This is the author’s black-box reconstruction, not verified access to Instinct’s code.
- Anthropic: The components of agent memory — memory layers, source-based correction and keeping injected context small. This prompted our November review.
- TypeSafe: Introducing System One Models & Jev — structured probabilistic decisions; the model family we explored for relevance scoring.
- Wes Sander: Markdown Is All You Need and Markdown Agent Memory — a comparison point for readable provenance, supersession and bounded recall, not a package we claim to have adopted.
- fast-jev-compaction — an approach we investigated for retaining useful context. Conversation compaction and durable memory solve different problems.
Related research
- Lin et al.: Sleep-time Compute: Beyond Inference Scaling at Test-time — offline work before future queries. Background for the maintenance tradeoff, not evidence that our nightly worker is operational.
- Anthropic: Effective context engineering for AI agents and Chroma: Context Rot — background on context selection and long-input degradation. This differs from a harness truncating text before inference.
The failure cases and numbers in this article are our own retrospective audits and small experiments, not results from those papers. The 26-case comparison used three answers per case and two automated judges, with no human validation. The reported interval crosses zero. We did not isolate a dollar cost or a causal attention effect for the original giant-file design, and no fresh production check is implied by this article.
All generated images are conceptual. The final architecture and walkthroughs describe intended contracts; the implementation coverage panel marks what was observed and what remains unproven.


