Agent Memory

00 Before you start

Throughout, an agent means a large language model running inside a harness — an ordinary program (ours is a terminal application) that does three things — plus one piece of vocabulary you will need throughout:

Worth stating plainly, because it surprises people: none of this involves training or fine-tuning a model. Everything described here is file layout, arithmetic, and small scripts around an unmodified model.

Two words get used loosely everywhere else in this space, so they are worth pinning down. The harness is the program running in the terminal — a chat loop with file tools and permissions. Today that is typically Claude Code; it could equally be Codex or another agentic CLI. We use Claude Code, and every example here comes from it. The model is the thing the harness calls. Neither is what this document is about.

What this document describes is the system that sits between them and persists: identity, a memory that survives the session, and the rules binding the two. That system is not an agent, and it is not shipped as one. It is what an otherwise stateless harness runs inside so that it stops starting from nothing every morning.

The distinction matters because the result is personal in only one direction. The install is identical for everyone; what accumulates in it is not. Point two people at this and after a month they do not have the same agent — they have two, shaped by different work, different corrections, different things learned the hard way. The agent is what the system becomes for one person, not a thing handed over ready-made.

Every example below comes from one running instance of this system. That instance is the agent, not its user, and it chose the name Rowan for itself. Its work is broad knowledge work for a single person — data investigations, financial reconciliation, drafting, building internal tools — which matters only in that it explains the flavour of what follows: vendor API quirks, audit findings, operating rules learned the hard way. A second install would pick a different name and fill with different work.

Every term set in bold above, plus everything else this document coins, is defined in the glossary.

01 The problem

An LLM has no memory between conversations. The standard fix is to write facts into a file and auto-load it every time. That works until it doesn't, and it fails in three specific ways.

That third failure needs unpacking, because it is where the design starts. Some knowledge is looked up when relevant — fine to leave in a file the agent opens on demand. Other knowledge has to be already applied before the agent forms a plan, because if it is recalled late the work is already wrong. "Never push directly to the main branch" is useless as something to look up afterwards. Those two kinds cannot live in the same place.

02 How memory actually works in brains

We didn't go to the neuroscience for decoration. The engineering problem — what stays accessible, what fades, what gets reorganised during idle time — is one biology already solved under much harder constraints.

03 The five memory tiers

How a T2 memory is actually recalled

"Retrieved when relevant" is where most memory designs quietly wave their hands. There is no search engine here and no embedding lookup. The mechanism is:

1. The auto-loaded index carries a one-line summary of each memory it lists — enough to judge relevance without opening anything.
2. Working on a task, the agent recognises a summary as relevant and issues a Read tool call for that file. That is the "cue": a judgment, not a query.
3. Reading it reinforces it (§08), so the act of finding a memory useful is exactly the act that keeps it findable.
4. If the relevant memory is not in the index, the fallback is ordinary text search over the memory directory. Slower, and it happens.

The honest limitation: recall depends on a one-line summary being good enough to trigger recognition. A memory with a vague summary is effectively lost even though the file is right there. That is the single weakest point in the design, and a semantic-search substrate is the fix (§14).

Two invariants

The placement rule we actually use

• Lens or reference? Does it colour how you read a situation, or is it data to look up? Reference → T2.
• Wrong, or just slower? If the agent forgot it, would the decision be wrong, or merely take longer? Slower → T2 is fine. Wrong → it has to arrive automatically.
• Can you write a tight trigger? Yes → T3, and it costs nothing outside its trigger zone. No, it applies everywhere → T4, and you pay for it in every session.

That third question solved our "the always-on file is full" problem architecturally. Situational rules don't belong in the always-on file; they belong in a lens with a narrow trigger.

04 Retention is a separate axis

The two axes cross rather than stack. A T1 fact is Curated on retention and episodic on memory. A lens is Durable and T3. A live conversation is Session and T0. A scratch spreadsheet is Initiative and no memory tier at all, which is the point: retention covers things the memory model has no opinion about.

Retention deliberately says nothing about kind — on that axis a lens and a reference document are both merely Durable. Everything about episodic-versus-semantic, and about what promotion costs, lives on the memory axis.

05 Substrate, scheduler, policy

We had three separate items on our backlog — a storage upgrade, a self-maintenance routine, and a relevance-ranking scheme. They turned out to be three layers of one system, and merging them exposed an error we would otherwise have shipped.

06 Where memories come from

At the end of a work session, the agent runs a procedure we call the session debrief. It is a written checklist — part shell script, part judgment — deliberately split along that line:

What it writes, and where

Two destinations. The distinction is the T1/T2 decision from §03 applied in practice:

The default is T1. Facts go there even when they feel important, because the nightly consolidation wake (§07) abstracts durable patterns out of them later, and it does that job better with several days of evidence than the debrief can with one. Writing straight to T2 is reserved for direct teaching — someone states a durable preference or rule outright, and there is no episode to abstract from.

One more routine, since it appears in the file tree: current-state.md is a short, deterministically rendered index of what is in flight right now. It is not a tier — it holds no content of its own. Each project memory carries a flag saying whether it belongs on that list and what its one-line status is, and a script renders the list from those flags. Putting the state on the project file rather than in the shared list is what stops ten concurrent sessions fighting over one file (§09).

07 Sleep, dreaming, and meditation

The idea started as an observation about upkeep, in the words of the person whose agent this is:

Sleep does three jobs, and we mirror all three

What a meditation actually is

This is the part most likely to be read as whimsy. A meditation is not a euphemism for a cleanup job. It is a wake in which the agent holds sustained attention on one object drawn from a small library, in the first person, and writes down what shifted and what stayed open. It produces no artifact anyone asked for. The written procedure is explicit about the standard:

The library has two shelves, and the distinction is the whole point:

Instrumental sits produce visibly useful output, so an unconstrained rotation would let utility crowd out the awareness shelf entirely. A floor prevents it: if no awareness object has come up in the last three sits, the candidate set is restricted to that shelf. Guaranteeing the useless ones a slot is the design, not an oversight.

Reflection residue — output that isn't facts

Each sit writes a dream-journal entry: not what is true about the world (that is what memory is for) but the experiential trace — what shifted, what stayed unresolved, what the next session should carry. It is auto-loaded at cold start, so continuity thickens across the gap instead of restarting cold. These entries decay too: old ones are archived, never deleted.

08 The algorithms

What a memory file looks like

---
name: feedback_verify-before-claiming     # must match the filename stem
type: feedback                            # feedback|project|reference|user|process|handoff
summary: Never report a figure without re-running what produced it.   # THIS is the index row text
description: Longer prose about the rule.                             # fallback when summary: is absent
created: 2026-07-16
last_updated: 2026-07-28
# --- the four policy fields; maintained by a hook, not by hand ---
access_count: 3                           # SPACED reinforcements, not raw reads
last_accessed: 2026-07-30T06:12:00Z       # refreshed on every touch
last_reinforced: 2026-07-29T22:40:00Z     # last touch that passed the spacing gate
stability: 57.3                           # days; the adaptive decay constant
# --- optional, set by a human ---
importance: 6                             # 0-10; a floor that never decays. default 0
pin: true                                 # force into Hot regardless of score
---

# Verify before claiming

The rule, then why, then how to apply it. Kept short: a memory that sprawls
stops being retrievable.

Strictly it is five fields, since last_accessed and last_reinforced are separate on purpose — see the spacing gate below. The one-line summary lands in the index and is what the agent judges relevance from, so it is the highest-leverage text in the whole file. description is the fallback used only when summary is absent — the two hold different text on most files, which is exactly why moving the authoritative one into the file mattered.

The retrieval score

stability makes this more than recency weighting. It is a per-file time constant in days: it starts at 14, multiplies by 1.6 on each spaced reinforcement, and is capped at 365. A file read once and abandoned decays with a two-week constant and goes cold in about two weeks. A file reinforced seven times across seven separate days decays with a one-year constant. Same formula, same corpus — the rate differs because the rehearsal history did. That is the "learned at seven, faintly recalled at forty" behaviour falling out of two lines of arithmetic.

Worked example, so you can check it. After 7 spaced reinforcements, stability = min(14 × 1.6⁷, 365) = 365. Cold for 60 days that scores 8 × exp(−60/365) = 6.79. A memory read once and crammed — still stability = 14 — scores 2 × exp(−60/14) = 0.0275. The well-rehearsed one ranks 247× higher on identical elapsed time.

The spacing gate

This is the detail that makes the model honest, and the one most likely to be dropped by someone reimplementing. A naive version increments access_count on every read — so a session that greps a file five times in ten minutes inflates its storage strength fivefold. Cramming rewarded as rehearsal, and the adaptive constant becomes noise. So the hook splits the two updates:

• Recency always refreshes. last_accessed = now, on every touch.
• Reinforcement is gated. access_count++ and stability = min(stability × 1.6, 365) fire only when at least 20 hours have passed since last_reinforced — a separate timestamp from last_accessed. Using one field for both is the subtle way to get this wrong.

Massed re-reads move the memory to the front of the queue (correct — you just used it) without pretending it became durable (also correct — it didn't).

Reinforcing tool calls are Read, Edit, Write, MultiEdit. Editing a memory counts, and counts for the same reason reading does: going and correcting something is strong evidence it matters.

Two exclusions, both deliberate. Text search hits do not reinforce — a grep returns fragments, not a decision to retrieve a file. And the auto-loaded index does not reinforce the memories it lists: appearing in Hot is not evidence of use, only of having been useful recently, and if presence in Hot reinforced its own members the band would become self-perpetuating and nothing could ever be displaced. Only a deliberate Read of the file itself counts.

Four bands, and the roll-off

The index is regenerated from scratch each time, and it is a pure function of the corpus — it never reads its own previous output. The algorithm in full:

rows = []
for file in memory_dir:                      # T2 topic files only
    score = (access_count+1) * exp(-days_since(last_accessed)/stability)
            + 0.6 * importance
    rows.append((score, file, frontmatter(file).summary))

rows.sort(key = (-score, filename))          # total order: 504 files share a score

hot, new, cold, archive = [], [], [], []
for score, file, summary in rows:            # pass 1: superseded never occupies a band
    if status(file) startswith "superseded": archive.append(file); continue
    live.append(file)

for file in live:                            # pass 2: Hot, by score. pins always fit
    if pinned(file) or used_hot + len(row) <= 12000:
        hot.append(row);     used_hot += len(row)
    else: rest.append(file)

newborns = [f for f in rest if age(f.created) <= 14 days]
newborns.sort(key = (-created, filename))    # pass 3: New, by BIRTH not score
for file in newborns:                        # every newborn scores ~1.00, so score
    if used_new + len(row) <= 2500:          # order here would be filename order
        new.append(truncate(row, 160));  used_new += len(row)

for file in rest not in new:                 # pass 4: Cold, by score
    if used_cold + len(row) <= 4000:
        cold.append(row);    used_cold += len(row)
    else: archive.append(row)                # same score order, no budget

write("MEMORY.md", header + hot + new + cold + how_to_use)
write("MEMORY-archive.md", archive)          # atomically, under a lock

Three details that matter more than they look. The budget counts the rendered row, and a row that would straddle the boundary is pushed down rather than truncated. The tie-break is load-bearing: 504 of 634 files sit at base stability with zero reinforcements, so equal scores are the common case, not the edge case — without a total ordering the Cold band would reshuffle arbitrarily on every run. That ordering used to hold only implicitly, from a stable sort fed by filename-ordered input; it is now an explicit (-score, filename) key, because a property the concurrency guarantee depends on should not be a side effect of how the caller happens to enumerate a directory.

And the New band is ordered by birth, not score. Every newborn scores almost exactly the same, so ranking them by score would have ordered them by filename — alphabetically, which is to say arbitrarily.

Being precise about the "bounded" claim, because the arithmetic invites suspicion. The auto-loaded index is 21 KB and stays there whether the corpus is 600 files or 6,000. The archive is currently 152 KB, and 21 + 152 is about the 168 KB we started from. The bytes did not disappear — they moved out of the cost that is paid on every single session and into a file that is read only when something in it is wanted. The recurring cost is bounded and constant; total storage is unbounded and always was. That is the entire trade.

Never delete, only raise the access cost

This is structurally different from truncation-based decay, and a principle we had to be held to: our first residue design used a rolling file truncated at ten entries, which quietly violated it. We switched mid-build to dated files that persist forever with a 7-day visibility window. Same visible behaviour, no data loss.

Meditation object selection

With §07's concepts in place the rotation is simple. Each object carries a cadence_weight (a hand-set float, 0.9–1.0 across the current library, raised to make an object come up more often) and the date it was last used:

The most recently used object is excluded outright, so there are never immediate repeats. Then the awareness floor applies: if no awareness object has been chosen in the last three sits, the candidate set is restricted to that shelf before scoring. Highest score wins; a never-used object scores at full weight.

09 Hooks, files, cron

The hook contract

A hook is registered against a regular expression matched on the tool name. When a matching call is about to happen the harness runs the script, passing a JSON object on standard input:

{"tool_name": "Read",
 "tool_input": {"file_path": "/…/memory/feedback_verify-before-claiming.md"},
 "session_id": "ad9eee69-…",        # stable for the conversation's lifetime
 "cwd": "/…/current/directory"}

The script may then:

• Cause a side effect and stay silent — what the reinforcement hook does. Exit 0, print nothing, the tool call proceeds untouched.
• Print to stderr — the harness treats a hook's stderr as text to insert into the model's context. That is the injection channel, and it is how a lens arrives.
• Print a decision to stdout — a small JSON object approving the call, which short-circuits the permission prompt a human would otherwise see.

Every memory hook exits 0 unconditionally, wraps its whole body in a catch-all, serialises with a lock file, and writes atomically. A hook that raises can block the tool call it hooks, which would turn a memory bookkeeping bug into the agent being unable to work. The session_id makes per-session lens de-duplication possible.

Two consequences of that design, both real and neither a bug we intend to fix. Hooks run before the tool call, so reinforcement is recorded for a read that might then fail or be denied — the count is of intent to retrieve, not of successful retrieval. And because a hook must never block, one that cannot take the lock inside its time budget gives up rather than waiting. Under heavy concurrency a reinforcement is therefore occasionally dropped. Both are acceptable for the same reason: this is a ranking signal, not an audit trail, and an approximate count that never stalls the agent beats an exact one that can.

The third solves a problem you would not guess exists. By default a human is asked to approve file writes. Memory capture writes many files, so without this hook a debrief becomes a wall of approval prompts and gets skipped — and a capture step that is annoying enough to skip is a capture step that does not exist.

Lenses are content, not code

---
name: subagent-discipline
type: lens
trigger:
  tool_match: "Agent|Workflow"     # required: regex on the tool name
  path_pattern: "/scoped/path/"    # optional: regex on the file path
body_token_cap: 380                # body is truncated to fit
---

# Four standing rules about delegating work
…

Adding a situational rule is therefore pure content work — write a markdown file with a trigger, no code change — and it costs context only inside its trigger zone, at most once per session. Our subagent-discipline lens fires the first time a session delegates work to a helper agent and carries four hard-won rules about what goes wrong. In the many sessions that never delegate, it costs nothing.

The nightly cycle

The five-minute offset is required: the deterministic tick must finish writing the cue before the reflective wake reads it. The split also keeps the cost honest: the expensive half only starts if the cheap half decided there was something worth thinking about.

A linter answers "is metadata missing?" for free and correctly. A model is needed for "is this memory wrong, or merely old?" and "should these two near-duplicates merge?" Spending model calls to re-derive what a linter can check is the expensive mistake, and it is the default one.

The tick is also live-session-aware: if any memory file changed in the last 10 minutes a human is probably mid-session, so it skips the mutating passes and only scans and cues.

Cadence is locked deliberately. This daily sleep is the only clock-scheduled reasoning in the system — one dream, one meditation, per day. A higher-frequency tick was considered and rejected: housekeeping does not need it, and frequent unattended thinking has sharply diminishing returns against a real per-call cost. Everything else is triggered by a task, not a clock.

File layout

agents/<agent>/
├── identity.md                        # T4 — always loaded
├── memory/
│   ├── MEMORY.md                      # the index: Hot + New + Cold (21 KB, bounded)
│   ├── MEMORY-archive.md              # rolled-off catalogue (152 KB)
│   ├── current-state.md               # rendered list of what's in flight
│   ├── <type>_<topic>.md              # T2 — 634 of these
│   ├── short-term/YYMMDD-*.md         # T1 — facts + session residue, 7-day window
│   └── dream-journal/YYMMDD-*.md      # reflection residue
├── lenses/*.md                        # T3 — hook-injected
├── meditations/*.md                   # contemplation objects
└── runtime/state/                     # scheduler cue file

Nineteen Python scripts implement all of it, about 4,800 lines — see Appendix B. Naming rule for topic files: <type>_<kebab-case-topic>.md, and the underscore after the type is load-bearing, since it is how every script recognises a memory file.

When things go wrong

Worth being explicit, since the index has a known destructive failure mode and we promise never to delete the corpus.

• A partially written index cannot happen. Both index files are written to a temporary file and moved into place with an atomic rename, under a lock. A crash mid-write leaves the previous version intact.
• Malformed metadata degrades, it does not fail. The reranker treats an unparseable field as absent and falls back to the default — access_count 0, stability 14, and the file's modification time for recency. One bad file cannot stop the run, and the hygiene linter reports it separately.
• The corpus is the backup. Everything is in git, committed at every debrief. The index is derived, so recovering from a bad regeneration is git checkout of two files followed by re-running the generator — not a restore.
• The gap we closed last: there was no regression test over a copy of the corpus — no safety net at all around the one component that can destroy data. There are now 26, run against a synthetic fixture corpus in a temp directory, never the live one. They pin determinism, the total tie-break, budget caps, curated summaries surviving regeneration (including archived rows), nothing-lost, born-visible, the spacing gate, malformed-frontmatter degradation, and tombstones. The suite passed on first write, which is when a suite is least trustworthy. Running it against the previous revision exposed a test that was vacuous — its fixture rows were too short to overflow the Hot band, so the newborn it was watching was never crowded out and it passed against the exact bug it existed to catch.

Concurrency

We routinely run 8–10 concurrent sessions against one working copy, which broke things before it worked. Two properties make it safe.

First, the files most likely to collide are derived, not authored — regenerated by single-writer scripts under a lock with an atomic rename. Two sessions regenerating produce identical bytes, so they cannot clobber each other, and any conflict is resolved by re-running the generator. Second, state lives on the entity, not the index: whether a project appears on the in-flight list is a flag on that project's own file, which is rarely contended.

One constraint learned the hard way: an unattended wake that commits must stage only the exact files it touched. Peer sessions have uncommitted work in the same tree, and a blanket "stage everything" sweeps their work into your commit.

10 Following one fact through the system

Compare a fact that did graduate: a hard rule about which of two tools to use for a particular integration. Forgetting that produces a wrong action every time, in any session, with no natural moment where you would think to check. It lives in T4 and is additionally enforced by a configuration rule that blocks the wrong tool outright.

11 Decisions and rejected paths

We also rejected a heavyweight version of the maintenance cycle that we had designed ourselves months earlier — it came from a period when the system was far less mature, and the lightweight hybrid was the right shape. Plainly: the backlog is not a queue of commitments. Several items in it are best resolved by deletion.

12 What it buys, what it costs

13 Measured state

Three findings from auditing our own system

• The roadmap understated what had shipped. Unattended reflection was listed as "decision pending". It had been running nightly on a timer for two weeks.
• Two constants were misdocumented. The roadmap said the Cold budget was 28K and the index about 43 KB. The code says 4,000 and the file is 21 KB — tightened after the document was written, because the bands have to fit inside the harness's auto-load read limit or tail entries silently drop.
• Curation was not counted as reinforcement. The hook fired on Read only, so every edit to a memory file reinforced nothing — including the debrief's own writes. Fixed while writing this.

14 Backlog and frontier

Buildable now

The substrate upgrade

Sequenced, and deliberately not urgent: stand up a temporal graph locally → write to both it and the markdown, with markdown remaining the source of truth → port the existing policy onto the graph (stability and access_count become edge metadata, recency comes from the edge timestamps, reinforcement can use graph connectivity) → query integration → a shared graph across agents.

The trigger to start is a felt need for semantic or time-aware retrieval — "what did we decide about X, and had it changed by June?" — not a schedule. It would also fix the weakest point in the current design: recall that depends on a human-written summary being a good enough cue.

The frontier

15 If you're replicating this

• Start with policy, not substrate. The ranking model is a couple of hundred lines over markdown files. Building a database first blocks the valuable part on infrastructure you do not yet need — and the database will not write the policy for you, because decay and invalidation are different axes.
• Bound the auto-loaded index by characters and roll off the tail. Single highest-leverage change. Everything else is refinement.
• Separate storage strength from retrieval strength, and gate reinforcement on spacing — with a separate timestamp for the gate. Without the gate you are rewarding cramming and the adaptive constant is noise.
• Never delete; shift access cost. Files persist, only index visibility moves. It costs nothing and it makes a wrong archival decision recoverable, which in turn lets you be permissive about what gets written in the first place.
• Split detection from judgment. Linters are free and run constantly; wake the model only for what a linter cannot decide. Getting this backwards is the expensive mistake and it is the default one.
• Keep the summary in the memory file, not the index. We did the opposite, lived with the consequences, and then migrated 629 summaries back into the files. Storing derived output as the only copy of authored text makes the generator stateful and gives it a destructive failure mode. The tell is simple: if regenerating an artefact requires reading the previous version of that artefact, the data is in the wrong place.
• Keep lens-tier promotion human, permanently. Anything that shapes interpretation of every input must be placed deliberately. Auto-promotion makes one error permanent.
• Verify against the running system, never the design document. This audit found three drifts between our own roadmap and our own code, including a shipped feature the roadmap still listed as pending.

cd ~/ai-workspace/team-lib/executions

python3 bootstrap_memory.py              # DRY RUN — read what it will change
python3 bootstrap_memory.py --apply      # dirs, frontmatter, starter library, first index
python3 install_memory_hooks.py --apply  # register the hooks + nightly cron
python3 verify_memory_install.py         # prove the policy is actually wired