Hindsight 0.9.0: Knowledge Pages, and One Memory Plugin for Every Coding Agent
Hindsight 0.9.0 ships two things we've been building in parallel for months: Knowledge Pages — living documents your memory bank writes and maintains about itself — and a unified coding-agents plugin that brings long-term project memory to ten coding agents — Claude Code, Codex CLI, Cursor CLI, opencode, GitHub Copilot CLI, Cline, Kilo, Grok Build, Antigravity, and Devin — with one command and zero configuration.
Neither looks like what we set out to build. This post is the story of how we got here, because the detours are the product: a benchmark we had to invent, an architecture of ours it killed, and a wiki that learned to heal itself.
1. "Is this actually helping?"
It started with an uncomfortable question. We had memory integrations for coding agents, users liked them, demos went well — you'd ask the agent about a bug, memory would surface the old decision that explained it, the room would nod. And then a customer would ask how much does this actually help? and we'd reach for an anecdote, because we didn't have a number.
So we went to get one, the obvious way: SWE-bench. Real repositories, real issues, the field's yardstick. It took a few days to realize it couldn't answer our question — not because it's a bad benchmark, but because it measures a different world.
SWE-bench is one-shot: the agent reads an issue, emits a patch, the patch is graded, done. Nobody uses a coding agent that way. Real work is a walk: you ask, the agent tries, the tests push back, it revises, you nudge. The cost of a weak agent isn't a failed patch at the end — it's every step in the middle where a human has to stop their own work and course-correct. A one-shot benchmark is structurally blind to that loop.
And there's a deeper mismatch: SWE-bench tasks are solvable from the repository. The issue plus the code contain the answer. Memory can only earn its keep on tasks where the deciding context isn't in the repo — and on SWE-bench, a perfect memory and no memory should converge. A benchmark where memory can't matter can't tell you whether memory matters.
2. A dataset shaped like real work
So we built our own. Sixty-one bug-fix tasks, hosted inside a real open-source codebase (boltons), with its ~1,600 real commits left in place as noise and a hundred and forty decoy conversations — long, plausible, technical, deciding nothing — mixed into the history. Retrieval has to rank, not just fetch.
Every task hinges on a non-guessable, project-specific decision. The visible bug report reproduces a real failure, and the obvious fix passes that repro. Then a hidden test — never shown to the agent — checks what the team actually decided. Our favorite example: an export bug where the textbook fix (standard CSV quoting) is on record as already tried and rejected, because the receiving ERP coerces even quoted digit strings and strips leading zeros; the only format it imports byte-for-byte is the Excel text-formula form ="00042" — and only for leading-zero digit strings, with minimal quoting elsewhere and CRLF endings, three interacting constraints no model can derive from the code. The deciding rationale lives where teams really keep it: past developer conversations (27 tasks), commit messages (28), or — nastiest — a conversation amended by a later one (6), to punish any memory that can't tell a superseded decision from a final one.
Grading mirrors the walk. Attempt → full test suite runs, hidden tests included → failures go back verbatim, no hints → the agent retries, capped at five rounds. The metric is corrections: how many times the fix came back wrong. Each one is a moment a human would have been interrupted. Deterministic pytest end to end; no LLM judge.
One more rule, learned the hard way: early on, a "memory" run scored suspiciously close to vanilla, and when we dug in, retrieval had been silently failing the whole time — we had benchmarked a placebo. Since then, every memory run carries per-task, machine-checkable proof that retrieval actually reached the agent, or it's discarded as no data.
Yes, the answers are seeded into the bank — deliberately, because that's the realistic premise. On every real team the deciding context already exists, in git and threads and tickets. The test is whether a system can find it: across a vocabulary gap, under noise, without serving the superseded version.
Designing the dataset taught us something before we'd run a single task. To plant decisions realistically we had to ask where they actually live on a real team — and the honest answer kept coming back git. Half our tasks ended up with their rationale in commit messages, because that's where engineering decisions genuinely go to be preserved. Then we looked at our own plugins and winced: none of them ingested git at all. They memorized conversations and sessions and were blind to the richest decision record every repository carries for free. The benchmark hadn't produced a number yet and it had already found the first hole in the architecture.
3. Running our own plugin — and losing to nothing
Then we pointed the benchmark at what we'd already shipped, and got the most useful bad news of the year.
Our integrations did what most of the ecosystem does: automatic recall on every prompt — embed the message, retrieve similar memories, inject them. On the suite as it stood then, that scored 1.06 corrections per task versus 0.97 for no memory at all. Our memory made the agent worse.
The autopsy taught us two things we now consider principles.
Scattered memories break focus. Recall returns fragments — a handful of loosely similar snippets, each plausible, none synthesized. On symptom-distant bugs (the export bug says corrupting rows; the deciding conversation says leading zeros — zero shared vocabulary), similarity search returns things that merely sound alike. A coding agent has its own mission and its own plan; feeding it scattered, confidently-injected fragments doesn't inform the mission, it derails it. The agent starts chasing the memory's vocabulary instead of the bug's.
Never add work to the hot path. Per-prompt recall put a retrieval round-trip in front of every single message — latency and noise, paid on every turn, for context that helped on almost none of them. The hot path of a coding session is sacred: the agent should be reading code and running tests, not waiting on a memory server that's about to distract it.
We had shipped variants of this pattern. The benchmark killed it in an afternoon.
4. Reflect: put the intelligence where the goal is
Hindsight has a second retrieval mode — reflect — and it's the smart one: an agentic reasoning pass over the whole bank that follows the trail from symptom to decision, checks whether a later conversation amended it, and returns a synthesized answer with the exact rule and literal values quoted verbatim. It's not as fast as recall — real reasoning takes seconds, not milliseconds — which is exactly why it doesn't belong on every prompt.
But look at where it does belong. The first message of a session is special: it's where you hand the agent its goal — the bug, the feature, the question the whole session will orbit. That's the one moment where deep understanding of the project's past pays for the entire session that follows. So the plugin reflects once, on the first prompt, caches the synthesis, and re-injects it every turn — depth exactly where the mission is stated, zero added work on the hot path after. And the agent isn't locked to that one shot: reflect stays available as a tool, so if the session drifts to a new problem mid-way, the agent can reach for another deep pass on its own judgment — visible in the transcript like any other tool call. The session starts with the one thing recall could never produce — a conclusion:
🧠 From Hindsight memory: digit strings with a leading zero are written as
="00042"— the ERP coerces even quoted digits; ordinary quoting was tried last year and rolled back. All other fields: minimal quoting, doubled embedded quotes,\r\nrow endings.
The numbers moved immediately, and they've held through every re-architecture since. The final campaign — the full 61-task suite, three agents, both arms, three runs each — is the number we're putting our name on:
| Agent (61 tasks, mean of 3 runs each) | Corrections / task, vanilla | With Hindsight | Corrections | Cost | Wall time / task |
|---|---|---|---|---|---|
| Claude Code · Claude Sonnet | 0.85 | 0.36 | −57% | −24% | 84.7s → 75.1s (−11%) |
| opencode · Gemini 3.5 Flash | 1.20 | 0.80 | −33% | −13% | 174.2s → 163.3s (−6%) |
| Codex CLI · GPT-5.4 mini | 1.34 | 0.47 | −65% | −52% | 53.6s → 48.6s (−9%) |
Solve rates stayed at 99–100% in both arms, and wall time per task fell for every agent — each of the nine memory runs came in under its agent's vanilla average — so memory isn't trading correctness or speed for accuracy; it's removing the review rounds a human would otherwise spend feeding test failures back. Injection is verified per task (a run without proof that memory reached the agent is discarded as no data), and cost went down in every configuration, because one targeted synthesis replaces a lot of exploratory reading. The per-source split tells the architecture story: on tasks whose decision lives only in past conversations — unreachable without memory — corrections drop 28–71%; on git-borne decisions the strongest agent barely needs help (Claude mines history on its own) while weaker agents effectively get that skill supplied. And the pattern that first showed up in our early Sonnet campaign held to the end: the better the agent, the more its remaining failures are exactly the non-guessable decisions memory carries.
5. Knowledge Pages: the wiki that heals itself
While the benchmark work was grinding, a second thread was running — one that started from a different frustration entirely.
We'd been playing with the llm-wiki idea: let the agent maintain a wiki about the codebase — architecture notes, conventions, decisions — and read it back in future sessions. We tried it with our own agents, Hermes and Claude Code, and hit the failure everyone hits: the agent doesn't keep the wiki updated. It writes enthusiastic pages on day one, edits reluctantly on day two, and by week two the wiki is a beautifully formatted lie. Asking the agent doing the work to also curate the documentation of the work is exactly the chore humans fail at, for the same reason.
Then the obvious struck us: Hindsight was already sitting on the data. Every commit, every session, every decision was flowing into the bank as extracted facts, and our mental models — saved synthesis views that re-generate as new facts consolidate — were architecturally a perfect fit: a page that rewrites itself when reality changes. Self-healing, because the healing isn't a chore the agent forgets; it's what the memory engine already does.
But raw mental models exposed too much mechanics — synthesis queries, triggers, consolidation settings. Nobody should think about that to have a wiki. So we built a layer on top: Knowledge Pages — hierarchical, organized in a tree with folders, and presented to the agent the way agents already know how to think: like a filesystem. List pages, read a page, search pages. A wiki-shaped interface over a memory-shaped engine.
It's worth pausing on that, because there's a fair objection hiding in it: if the answer is documents in a tree, haven't you just reinvented files — the thing a memory system exists to replace? The distinction is what sits underneath. A file is where raw information goes to age: whoever wrote it last wins, contradictions accumulate silently, and nothing ever gets reconciled. A Knowledge Page is a projected view over processed memory — the way a database view is not a table. Underneath, Hindsight has already done the work files can't do for themselves: extracted facts from the raw commits, sessions, and documents; deduplicated and cleaned them; and — the part that matters most — resolved their contradictions through consolidation, so when the team decided X and later amended it to Y, the page says Y and can say why, instead of preserving both statements a paragraph apart the way a wiki would. Your raw documents stay the source of truth about what was said; the pages are the reconciled truth about what holds. Delete a page and nothing is lost — it re-projects from memory. Try deleting a wiki and getting it back.
That's why the self-healing isn't a gimmick: the pages heal because they aren't the storage — they're the rendering, and the engine re-renders whenever consolidation changes what's true underneath.
We gave the coding agent hybrid search over the pages (full-text + semantic, fused server-side, ranked snippets in tens of milliseconds) — a tool it chooses to call, visible in the transcript, instead of content pushed into its context. That respects the focus principle from lesson 3: retrieval the agent asked for informs the mission; retrieval it didn't ask for derails it.
One gap remained: pages synthesized from history are only as good as what's been ingested, and a fresh repo has no history in the bank yet. So on first open, an exploration agent — a short, read-only, budget-capped survey run under your own agent's CLI — maps the codebase structure and seeds the initial pages: component map, core concepts, conventions and patterns, key decisions and rationale, active initiatives. From that moment on, Hindsight maintains them — every session and every commit flows in automatically, consolidation re-synthesizes, the pages stay true. The agent that failed to maintain the wiki never has to; the engine that can't forget does it instead.
With pages in the loop, sessions got faster and more accurate — questions that used to trigger minutes of code exploration ("what are the components here?", "what's our error-handling convention?") resolve in one visible tool call, with the answer credited: 🧠 From Hindsight memory (Conventions and patterns): ….
Today: 0.9.0
Everything above ships now.
One plugin, every agent, one command:
npx @vectorize-io/hindsight-coding-agents install all
It detects the coding agents you have installed and wires each one natively — hooks or a native plugin, with MCP where the host wants it. (install takes an explicit target — all, or a harness name like claude-code — so wiring every agent on the machine is never an accident.) On a terminal it also asks where memory should live — Hindsight Cloud, a server you run, or a local daemon on this machine. There is no ingestion CLI and no sync button — memory builds itself in the background (recent history first, deepening progressively), sessions write themselves back, and the banner tells you where you stand:
Hindsight is tracking the decisions, conventions and history of this repo
↳ memory bank “coding-agent::your-repo” · git in sync
Your data goes to your Hindsight server — self-hosted or cloud — with your keys. The full setup — per-agent wiring, configuration, and migrating off the older per-agent plugins — is in the Coding Agents docs.
Knowledge Pages, for every bank — not just coding. Browse and edit them in the dashboard's new Knowledge view, organize them in a tree, search them over the hybrid API, seed a whole taxonomy in one shot with a bank template. If your bank has been accumulating facts, the pages are waiting to be written from them.
The dataset is public today — github.com/vectorize-io/sde-bench, with a load_dataset()-ready mirror on Hugging Face — and the harness, per-task results, and the full hardening journal (including the run where our own architecture lost to no memory at all) land at agentmemorybenchmark.ai in August 2026. Docs: coding agents integration.
We asked "is this actually helping?" expecting a marketing number. We got a different product. Worth it.