Skip to main content

Introducing Hindsight Memlake: serving 100M Memories at <100ms p99 on object storage

A new object-storage-native engine for Hindsight’s agent memory, built for fast retrieval, high-throughput ingest, and lower cost at scale.

By Nicolò Boschi
· 25 min read
Divergence chart as one bank grows from 4k to 3M memories, with Memlake as the flat central spine at about 90 ms p99 and $1.7k a month. Above the spine, Postgres recall p99 rises from 0.5 s to 930 ms at 50k and 5.3 s at 210k; below it, Postgres monthly cost grows to $7.4k at 210k and an extrapolated $73.3k at 3MDivergence chart as one bank grows from 4k to 3M memories, with Memlake as the flat central spine at about 90 ms p99 and $1.7k a month. Above the spine, Postgres recall p99 rises from 0.5 s to 930 ms at 50k and 5.3 s at 210k; below it, Postgres monthly cost grows to $7.4k at 210k and an extrapolated $73.3k at 3M

TL;DR​

  • Open-source Hindsight is 1.0. After a year in production, the architecture and the API are stable, and from here the API is a promise. It stays open source and keeps running on Postgres.
  • For enterprises: Hindsight Memlake. A new storage engine that keeps agent memory in object storage, in a bucket you control, searchable in milliseconds and built for a billion memories and beyond. It's part of Hindsight Enterprise, and most of Hindsight Cloud already runs on it.

Agent memory that learns, and only grows​

Hindsight is agent memory that learns. It doesn't store transcripts and search them later: it keeps turning what agents experience into knowledge, in layers that build on each other.

  • Facts. When an agent retains a conversation or a document, Hindsight extracts the facts in it, resolves the people and things they mention, and links related facts together.
  • Observations. In the background, consolidation merges facts into evidence-backed observations, and keeps them current as new facts confirm, refine or contradict them. This is where a bank stops being a pile of notes and becomes a view of its world.
  • Mental models and knowledge pages. You choose the questions that matter (how this customer likes to work, how this codebase is deployed), and Hindsight writes the answers and keeps them up to date as the bank learns. Knowledge pages are living documents that agents build and read the same way.

Agents use all of it in two ways. Recall runs four searches at once (by meaning, by keyword, through relationships and through time) and returns a ranked list that fits the context window. Reflect reasons over the bank to answer a question, reading mental models first, then observations, then raw facts. That design is why Hindsight is #1 on every dataset of the Agent Memory Benchmark: 94.6% on LongMemEval-S, 92% on LoCoMo, and 64.1% on BEAM at 10 million tokens.

Every one of those layers is more memory, and it never stops growing. Agents now write code, run support queues and operate systems, more and more on their own and around the clock. Nobody reviews every step, so they improve the way people do: by remembering what worked and what didn't. Unlike people, they never stop writing it down.1

  • An employee's agent writes about 68,000 memories a year.
  • An autonomous agent running ~500 tasks a day writes about 5 million.
  • A 10,000-person company with a thousand autonomous agents writes about 5.5 billion memories a year, and needs each one findable in under a second.
An employee's agent writes about 68,000 memories a year, an autonomous agent about 5 million, a 10,000-person company with 1,000 autonomous agents about 5.5 billion — and memory only grows, year after yearAn employee's agent writes about 68,000 memories a year, an autonomous agent about 5 million, a 10,000-person company with 1,000 autonomous agents about 5.5 billion — and memory only grows, year after year

At that scale, storage stops being a detail and becomes the constraint.

Object storage is all you need​

Today, Hindsight stores memory in Postgres (or Oracle 23ai), and for most users that's the right choice. In Hindsight Cloud, each organization gets its own schema; memories are rows with a vector index, a full-text index and a table of links between them. It's simple, it's familiar, and open-source Hindsight will keep defaulting to it.

But we run Hindsight Cloud across thousands of organizations, and at that scale we measured where a relational store stops keeping up with agent memory:2

  • Recall slows down as a bank grows. On a 4-vCPU primary, recall served 7.0 queries per second at 33k memories and 1.9 at 164k. Adding concurrency only added queueing.
  • Keyword search wasn't built for big banks. Postgres's built-in full-text search scores every matching row; on a 210k-memory bank one query matched almost all of them, even with Hindsight's default limit of the 16 rarest query words. Other keyword engines (vchord, pg_textsearch, pgroonga) would help with that part, but each is one more index to bolt on, and none of them changes the write and link costs below.
  • Writes hit the write-ahead log. Each memory produced about 190 KB of WAL across the vector, link and full-text indexes. Ingest topped out at 55–80 memories per second.
  • Many banks hurt too. One organization with 1,000 small banks sustained only ~2 recalls per second, because every query is planned over thousands of per-bank indexes.
  • Most of the bytes are links. In our hosted fleet, the table linking memories to each other was 65% of the database, and three quarters of it was index, not data.

Postgres isn't doing anything wrong here. The workload has an unusual shape: tiny records that only accumulate, four kinds of search over the same data on every recall, and every write visible the moment it lands. A general-purpose relational store pays for every memory when it's stored and again on every query, so a workload that only grows means a bigger primary, forever.

A horizontally scalable Postgres (Citus, Aurora Limitless, AlloyDB) would buy headroom, but it would scale the same design: every link still a row, every recall still a query across indexes that grow with the bank. So we built a store for this shape of data.

Introducing Hindsight Memlake​

Hindsight Memlake is our enterprise storage engine for agent memory, written in Rust for predictable memory use, no garbage-collection pauses on the recall path, and speed without giving up safety. It keeps everything in object storage (S3, GCS, or a certified S3-compatible store) and treats its servers as disposable caches. It plugs into Hindsight's database layer, so your agents, SDKs and API calls don't change at all.

It replaces Postgres only where memories live: memories, their links and indexes, the documents behind them, and mental models and knowledge pages (whose text is indexed for search) all move to the bucket. The operational side stays in Postgres: banks and their settings, the queue of background jobs, the audit log, and in Hindsight Cloud the account data. That data is small and relational, and it changes the way databases handle well, with updates, joins and transactions. Keeping it there lets Hindsight keep building features on plain tables, and lets each system do what it's best at.

Goals​

Before writing any code, we wrote down the properties Memlake had to have. Every change is still reviewed against them:

  • Object storage is the only source of truth for memory. Memlake itself has no database, no consensus cluster, no second stateful system beside the bucket. Durability, replication and availability come from S3 or GCS, and every byte of a tenant's memory lives in a bucket the tenant controls.
  • Durable before acknowledged, visible once acknowledged. When Hindsight confirms a memory is saved, it is already in object storage, and the very next recall from any server will find it.
  • Everything is incremental. Nothing on the write, indexing or recall path reprocesses a whole bank. A retain costs the same in a bank of a thousand memories or a hundred million, indexing works on new data only, and a recall makes a bounded number of round trips to object storage however large the bank.
  • One copy, four kinds of search. Each memory is stored once and serves vector, keyword, graph and time search. No separate vector database, search engine and graph store to keep in sync.
  • Compute is disposable. Servers hold only a cache. Losing one costs latency, never data, and capacity grows by adding servers.
  • Bounded by design. Caches and reads of not-yet-indexed writes run on fixed memory budgets. When writes outrun indexing, Memlake slows writers down with retryable errors instead of letting memory grow.

Architecture​

Memlake has four moving parts, and one of them is your bucket.

The bucket​

Each tenant gets its own bucket, and each memory bank is a prefix inside it holding three kinds of objects:

  • The log. Every write becomes one new, immutable file. The log holds recent memories until they're indexed.
  • Segments. Small, self-contained indexes over a slice of the bank, written once and never modified. Inside a segment, each memory type (facts, experiences, observations) has its own vector clusters, full-text index, entity index, time index and links.
  • The manifest and the head pointer. The only two objects that ever change: one lists the live segments, the other marks how far the log has been committed.

Everything else is a cache. And the two objects that change are only ever replaced with a conditional write: the store accepts the new version only if nobody changed it first. That is the whole coordination story. There's no database, lock service or leader beside the bucket, and Memlake's servers never talk to each other. Every decision that keeps your data safe (who claims the next slot in the log, when an index is published, what garbage collection may delete) comes down to a conditional write that the bucket either accepts or rejects. Because nothing is overwritten in place, a reader always sees a complete version of a bank, never something in between.

Writes​

A server writes each retain to the log with a conditional create, so two writers can never claim the same position, then advances the head pointer with another conditional write, and only then acknowledges. A Hindsight retain (new facts, replaced old ones, newly seen entities) is a single log entry, so it lands entirely or not at all. Writes don't wait for indexing: the next recall, from any server, reads the part of the log that hasn't been indexed yet.

Servers​

Hindsight talks to Memlake servers over gRPC. The load balancer sends each bank to the same server so its cache, in memory and on local NVMe, stays warm; since objects never change, that cache never needs invalidating. On a recall, the server checks the head pointer, reads only the blocks it needs from the segments, scans the unindexed tail of the log, and runs the four searches. Hindsight sends its ranking policy with every request, so the server ranks next to the data and returns only the memories that make the cut.

A bank's index lives on one server at a time, and Memlake grows by spreading banks across servers, which is how agent memory splits anyway: per agent, user or project. When a server disappears, another picks up its banks with no handover and warms its cache as it serves:

Indexers​

Indexers run apart from the servers that answer queries, and turn new log entries into segments:

An indexer folds new entries into a fresh segment, deriving links between memories as it goes, and publishes it by swapping the manifest. Compaction merges small segments in the background. Adding a hundred memories to a bank of a hundred million builds a hundred-memory segment, so indexing cost follows what's new, not what's stored, and the work that grows with the data happens here rather than at query time. Two indexers can work on the same bank with no lock: one wins the manifest swap, and the other's files are never referenced and later garbage-collected.

Archiver​

When backups are on, an archiver copies every bank's log, plus daily snapshots, to a separate bucket, ideally in another account with credentials that can't delete. A bank can be restored to any point in time, and every restore is verified before traffic moves to it.

The four searches​

All four searches read the same stored memories:

  • By meaning. Vectors are grouped into clusters (an IVF index), and a query scans only the closest ones. Each vector is kept twice: compressed with RaBitQ to about 60 bytes (for 384-dimension embeddings, 25× smaller than the original) for a fast first pass, and at full precision to rerank the best candidates exactly.
  • By keyword. BM25 over a tantivy index stored with each segment, including word segmentation for Chinese.
  • Through relationships. One step out from the best vector matches: to memories that share an entity, are close in meaning, caused or were caused by them, or happened right before or after.
  • Through time. Each segment has a time index; a query with a time window reads just that range.

What's exact and what's approximate: exact scores are comparable across segments, so merging each segment's reranked top results gives the same top results as reranking the same candidates in one place. Finding candidates within a segment is approximate, as with any cluster index, so in our tests we check that a search probing every cluster matches brute force exactly, and that on BEIR the compressed first pass finds the same top 10 as 8-bit vectors. The design expects a segment's search cost to grow with the square root of its size; our measurements below come close to that.

Verified, not just tested​

Because correctness comes down to a handful of conditional writes, the protocol is small enough to prove. The write, indexing and garbage-collection protocols are specified in TLA+, and a model checker explores every possible interleaving of them: many writers and indexers racing, servers crashing at any step, responses lost, processes stalling for arbitrarily long. Across all of it, it checks that:

  • An acknowledged write is never lost, and a position in the log is never reused.
  • Every query sees every acknowledged write, from any server.
  • Readers always see a complete index, never half of an update.
  • Garbage collection never deletes anything still in use.
  • Two indexers racing on the same bank stay safe with no lock between them.

The specifications are re-checked in CI on every protocol change, against deliberately broken variants the checker must catch, and recorded executions of the real engine are checked against them so the code can't drift from what was proven.3

Performance​

We measure Memlake two ways: head to head with Postgres through the full Hindsight API, which is what an agent experiences, and engine-level, straight against Memlake, to reach sizes the API benchmark can't. All recall numbers are without the cross-encoder reranker, which would otherwise dominate recall time on both sides.4

Fast as banks grow​

We ran the same workloads on the same development cluster, once on Postgres and once on Memlake, with the same data and benchmark driver:5

Recall throughput: Memlake 207, 106, 113 and 113 recalls per second against Postgres 19, 0.45, 4 and 2 across four workloads. One recall at p99: Memlake 106 and 113 ms against Postgres 513 ms and 5.3 sRecall throughput: Memlake 207, 106, 113 and 113 recalls per second against Postgres 19, 0.45, 4 and 2 across four workloads. One recall at p99: Memlake 106 and 113 ms against Postgres 513 ms and 5.3 s

Throughput is at 16 concurrent requests for single banks, and the highest steady rate for the multi-bank workloads. Postgres's big-bank latency is mostly keyword search;6 Memlake's 113 recalls/s is a ceiling of the test setup, not of Memlake.7

On a small bank, Memlake answers in a third to a fifth of the time and serves ten times the traffic. On a large bank the gap is two orders of magnitude, because Memlake's per-query work barely moves as the bank grows. Engine-level, growing one bank 30×, from 100k to 3M memories, made a recall about 6× slower under heavy load, close to the square root of the growth.8

Scales out across banks​

In our largest test, 100 million memories across 20 banks ran on 4 servers of 6 vCPU each, about a year of agents for 1,500 employees. Latency stayed flat from 1 to 200 recalls per second, and one 3M-memory bank answered in 71 ms at p50 and 90 ms at p99:9

Recall latency: one 3M-memory bank, one request at a time, p50 71 ms and p99 90 ms; 100M memories on 4 servers, p50 40–43 ms and p99 73–80 ms from 1 to 200 recalls per secondRecall latency: one 3M-memory bank, one request at a time, p50 71 ms and p99 90 ms; 100M memories on 4 servers, p50 40–43 ms and p99 73–80 ms from 1 to 200 recalls per second

Each server handles roughly 8–14 warm recalls per second per vCPU, so capacity grows linearly with servers. The largest single bank we've run holds 10 million memories, about two years of an autonomous agent working nonstop.

Writes scale too, though they're rarely the limit: every Hindsight retain goes through an LLM first, so how fast memories land is bound by the LLM, not the store. On its own, a single Memlake bank sustains 1,200–1,500 memories per second, flat as the bank grows: 15–27× what Postgres sustained. A write commits in about a quarter of a second, which Hindsight's asynchronous retain keeps off the agent's path.

The cold-start tradeoff​

Object storage makes warm reads cheap and first reads slow. The first time a server opens a bank, it loads the bank's index metadata from the bucket: 8.6 s at p95 for a 10M-memory bank, once per server rather than per query. Until the data blocks are cached, a query can take around a second at p99. Left to warm from traffic alone, a 1M-memory bank went from 1.7 s p99 to 74 ms once fully cached, so servers now preload what recall needs in the background whenever they open a bank or pick up a new index. Banks stay routed to the same server, so this happens rarely. And if you know a bank is about to get busy (a user logging in, an agent starting a job), you can send it one recall ahead of time: that opens the bank and starts the warm-up, so the real queries find it ready.

Retrieval quality​

Changing the storage engine didn't cost retrieval quality. On standard RAG datasets from the BEIR benchmark, with the same embeddings, Memlake's vector, keyword and hybrid search came out within 0.3% of an established vector database on every comparison.10

BEIR nDCG@10: on SciFact and NFCorpus, Memlake's vector, BM25 and hybrid search score within 0.3% of an established vector database; adding graph search lifts NFCorpus hybrid to 0.3645BEIR nDCG@10: on SciFact and NFCorpus, Memlake's vector, BM25 and hybrid search score within 0.3% of an established vector database; adding graph search lifts NFCorpus hybrid to 0.3645

Cost​

We priced a whole Hindsight cluster on each store with our internal sizing model: the memory store, the API, workers, disks and buckets. We left out the LLM and the GPUs for embeddings and reranking, which cost the same either way.11 The model gives each setup the smallest cluster that holds a year of memories and answers recall within 1 second at p99 at peak. One case needs care: when a single Postgres recall already takes longer than a second, as on big banks, no amount of hardware fixes it, because recall runs on one primary. There we size Postgres to keep up with traffic and report how slow each recall is.

Storage itself is cheap either way: about 4–5 KB per memory in the bucket, plus your documents, so a year of one employee's agent is roughly 350 MB, under a cent a month on S3. What moves the bill is serving recall. Here is a year for a whole company, with one bank per employee or agent. Almost all of the difference is the memory store itself:

Monthly cluster cost after a year, excluding the LLM and GPUs: 1,000 employees (68M memories) $3.2k on Memlake against $4.6k on Postgres; 10,000 employees (680M) $15.1k against $58.4k; 100 autonomous agents (490M) $11.1k against $146k; 1,000 autonomous agents (4.9B) $94k against $2.46M. One recall at p99: 0.35 s on Memlake in all four, against 2.4, 9.1, 5.2 and 11.1 s on Postgres, which misses the 1 s target every timeMonthly cluster cost after a year, excluding the LLM and GPUs: 1,000 employees (68M memories) $3.2k on Memlake against $4.6k on Postgres; 10,000 employees (680M) $15.1k against $58.4k; 100 autonomous agents (490M) $11.1k against $146k; 1,000 autonomous agents (4.9B) $94k against $2.46M. One recall at p99: 0.35 s on Memlake in all four, against 2.4, 9.1, 5.2 and 11.1 s on Postgres, which misses the 1 s target every time

Total memory count is not what makes Postgres expensive. Small banks cost about the same on either store, and a small deployment can be cheaper on Postgres. Memlake's advantage shows up as banks get big, and as there are more of them, which is where agents are heading. If your banks stay under ~50k memories each, Postgres is great, and it's not going anywhere.

Availability​

Hindsight Memlake is available today in Hindsight Enterprise, four ways:

  • Hindsight Cloud. Most organizations already run on Memlake. Nothing to set up.
  • A dedicated cloud instance. A single-tenant Hindsight that we run for you: its own cloud project, cluster and bucket, in the region you choose, on your own domain, with upgrades when you schedule them. Like running it in your cloud, without the operations.
  • Your cloud. Your own AWS or GCP account, on S3 or GCS, with your bucket's encryption and keys.
  • Your data center. On-premises, on your own S3-compatible storage, certified for the conditional writes Memlake depends on. We build this with design partners who make storage hardware; one runs the whole stack, models included, on its own flash.

Already on Postgres? A migration job moves an organization over, keeps every memory ID, and verifies the copy before switching. Book a call, or read the Memlake whitepaper and the architecture, guarantees and tradeoffs docs.

Hindsight 1.0​

We open-sourced Hindsight in December 2025. Since then it has passed 10,000 GitHub stars in April, 20,000 in August and 40,000 in September, and is now past 49,000; 1 million Python downloads in under six months; 263 contributors and a release every 4.8 days.

After a year in production, with more than 30,000 users on Hindsight Cloud, #1 results on every dataset of the Agent Memory Benchmark, and a stable architecture, we're calling it 1.0. Open-source Hindsight stays free, on Postgres, and keeps getting better; Memlake is there for the deployments that outgrow it. From here, the API is a promise.

What's next: Hindsight 2.0​

We're already building 2.0, and it'll get its own post. Three focuses:

  1. Memory for the models ahead. Some of the best memory designs weren't viable when we started, because models were too slow and expensive to run on every write. We're rebuilding core parts of Hindsight around newer techniques, including decision models: retain, recall, consolidation, reflect and knowledge pages. The goal is memory that costs less in LLM calls and compute, and gets more right at every step.
  2. Knowledge and experience in one memory. Hindsight is built for experience: what an agent has seen and learned. Teams also need knowledge (documents, wikis, runbooks), and 2.0 brings both into one system.
  3. Deep support for the agents that matter. General-purpose agents (Hermes, OpenClaw…) and coding agents (Claude Code, Codex, OpenCode, Pi…) are where memory pays off first. We're building deeper integrations and benchmarks that measure what memory does for them.

Try it​

Footnotes​

  1. Modelled. Employees average 10 conversations per workday; with LLM extraction, a typical conversation produces ~27 memories (measured). Autonomous agents run 500 tasks a day, every day. ↩

  2. Measured end to end through the Hindsight API on our development cluster, September 2026, except the link-table share (hosted fleet). The keyword cost comes from how Hindsight's Postgres recall builds its full-text query (it OR-joins the query's words, so on a large bank nearly every row matches). Open-source Hindsight already limits this by default — it keeps only the 16 rarest query words — and our runs had that limit on. Other keyword engines (vchord, pg_textsearch, pgroonga) are also supported; we'll keep improving this path for open-source users. ↩

  3. Model checking is exhaustive within a bounded scope (a few writers, indexers and log positions), which is where interleaving bugs live. It verifies the protocol as specified; the trace checks and a property-based test harness with injected storage faults tie that back to the real code. Details in the whitepaper. ↩

  4. All recall performance numbers in this post were measured without the cross-encoder reranker. With it on, reranking dominates recall time and is limited by the reranker's own GPU capacity, not by the store. ↩

  5. Same cluster, same data, same benchmark driver (no LLM), measured 2026-09-15. Not identical hardware: Postgres had a 4-vCPU primary shared with other development tenants. ↩

  6. Most of that Postgres time is keyword search: Postgres's built-in full-text search scores every matching row, and on this bank almost every row matched. This was measured with Hindsight's default limit of the 16 rarest query words on; a different keyword engine would bring it down further. Treat this row as the cost of the Postgres setup Hindsight ships today, not of Postgres at its best. ↩

  7. 113 recall/s is the ceiling of that test setup for a single organization, not a Memlake limit: Memlake's servers were far from busy, and the bottleneck appeared to be outside the store, most likely per-request billing on the organization's account row. Postgres's numbers are the highest rate it held before latency ran away. ↩

  8. Engine-level, August 2026: one 6-vCPU server, a local S3-compatible store, a realistic corpus (entities, tags, causal links, a 365-day span), 16 concurrent clients. Not fully warm (0.1–0.3 object-store round trips per recall). The unloaded 3M figure is from a later engine on GCS, one request at a time. ↩

  9. Engine-level benchmarks (gRPC to Memlake, no Hindsight API in front), July–August 2026. The 100M fleet and the 10M single-bank runs used object storage in another cloud, and the fleet ran on synthetic data; the 3M and √N results ran on a realistic corpus. Configurations and dates are in the whitepaper. ↩

  10. SciFact and NFCorpus, nDCG@10, bge-small-en-v1.5 embeddings for both systems, July 2026 engine. Being re-run on today's engine. BEIR has no entities or timestamps, so it tests that the storage engine doesn't cost quality, not what graph and time search add. ↩

  11. Modelled, not billed. Our internal sizing calculator prices a full GKE deployment (API, workers, Postgres, Memlake, disks, buckets, cluster fee) at GCP list prices after 12 months of growth, sized for a 1 s recall p99. It excludes the LLM and the embedding and reranker GPUs ($1.1–2.3k a month, the same on both). In the company chart, the rest of Hindsight (API, workers, login, cluster fee) is $0.6–2k a month on either store; everything else is the memory store. Recall times in the chart are the model's p99 for one recall (API plus store): for Postgres the measured single-recall time at that bank size (extrapolated past ~210k), for Memlake the budget of its default cache setting. Memlake servers are sized from a set of machine sizes, taking the cheapest that holds every recently used bank's index in its cache share. Agent assumptions as in the first footnote. Autonomous banks reach ~5M memories, past the largest bank measured through the API (151k), so their throughput is extrapolated; Postgres beyond ~210k memories per bank is extrapolated too. On the storage tier alone, our hosted fleet's shape is 2.2× cheaper on Memlake; the whole deployment, 1.5×. ↩