What's new in Hindsight 0.10.0

Hindsight 0.10.0 lets you retain a screenshot or a PDF in the same call as the prose around it, makes the request path several times cheaper, and shows you the exact prompt an operation would send before you spend a token on it. It also retires two endpoints that have been deprecated for several releases.

• Images and files are first-class retain content: content takes an ordered list of text, image and file blocks, and facts cite the attachment they came from.
• More throughput out of the same box: A 3.2x faster request path, concurrent embedding batches, and cheaper token counting.
• See the prompt before you send it: A preview endpoint for retain, consolidation and reflect, plus a retain prompt tester in the control plane.
• Recall: typo-tolerant tags, and labels you don't have to enumerate: A fuzzy tag resolve mode, and an open-vocabulary multi-text entity label.
• Take the synthesized layer with you: Knowledge pages and mental models travel in a document transfer.
• Every bank on one page: A banks overview with bulk delete is now the control plane's home page.
• Breaking changes: Removed bank profile and background endpoints, no curl in the Docker images, 503 instead of an unbounded queue under load, and one extension hook removed.

Images and files are first-class retain content

content now accepts an ordered list of text, image and file blocks instead of only a string. The extractor reads each attachment in the position it occupies — a chart between two paragraphs is read as the chart between those two paragraphs — and every read surface hands back the attachments behind what it returns. Provenance is per fact, not per chunk: a fact stated in the prose doesn't claim the screenshot next to it as evidence.

A plain string behaves exactly as before, byte for byte. Idempotency by content hash, update_mode=append, chunk-delta re-extraction and reprocess_document all keep working, because blocks are flattened at the API boundary into one canonical body and the bytes live in the existing file storage, content-addressed by SHA-256.

Recall carries them through. A recalled fact comes back with the attachments behind it, in the API response and in the control plane, so the screenshot a fact was read from is one click away from the answer instead of something you go looking for in the source document.

One migration ships with this (attachments, document_attachments); both tables are new and empty, so it's quick on any size of deployment.

More throughput out of the same box

A 3.2x faster request path. Both HTTP middlewares were rewritten as pure ASGI. Starlette's BaseHTTPMiddleware spawns a child task per request and pipes the response through a pair of memory-object streams; on cheap routes that machinery cost more than the endpoint. On a 2-CPU container, /health/live went from 2476 rps at p99 108ms to 7917 rps at p99 17ms, at lower CPU. Recall is unchanged — it's CPU-bound in the endpoint, nowhere near that ceiling — but every cheap route, health check and poll got a lot cheaper. Nothing was dropped: the HTTP metrics and the unknown-parameter reporting both moved rather than went away.

Embedding batches go out concurrently. Every remote embedding provider walked its batches in a plain loop, so a retain held exactly one embedding request open at a time however much text it had. Batching and a bounded fan-out now live in one place, results are concatenated in input order whatever the completion order, and the same TEI server sustains 903 texts/s at one in-flight request against 2080 at eight.

Cheaper token counting. Counting went through building a full list of token ids only to take its length — once per candidate fact, candidate chunk, source fact and reranker document. Token counting now runs on toktok, the BPE tokenizer we built for this as a drop-in replacement for tiktoken: it returns a count without materialising the ids, which takes the four counting stages of one recall from 34.2 ms to 5.0 ms, and a 77k-token document from 36 ms and 2.8 MB of allocation to 3 ms and effectively none.

Alongside those, recall stops building trace payloads when nothing asked for a trace, config resolution stops deep-copying the global config on every call, and the mental-model staleness check no longer walks the bank.

See the prompt before you send it

A bank's missions and strategies only mean something once you can see the prompt they land in. POST /banks/{id}/prompts/preview returns the messages retain, consolidation or reflect would send, in send order — no LLM call, no writes, runtime data replaced by a fixed placeholder. Messages come back as blocks, and the active ones concatenate to exactly the text that would be sent; a block for a setting you've switched off comes back marked inactive, so an unset mission is still visible in the place it would occupy.

The control plane adds a prompt tester for retain on top of it: edit the input, see the prompt, and run the extraction to see what facts come out.

Recall: typo-tolerant tags, and labels you don't have to enumerate

Fuzzy tag matching. Tag filtering is exact array containment, so a caller filtering on typsecript loses the memory tagged typescript before ranking ever runs — and no amount of better ranking fixes that, because the match itself has to tolerate the misspelling. A tag_groups leaf now takes an optional resolve field: leave it at exact and nothing changes, set it to fuzzy and that leaf's tags are matched against the bank's tags by trigram similarity instead of literally. That is the entire API change — no new config, no new response field.

It resolves typescropt→typescript, kubernets→kubernetes and user:alcie→user:alice, and deliberately doesn't resolve mango→mongo. One limit is worth knowing up front: similarity is length-sensitive, so a short tag has few trigrams and a single edit destroys most of them — kakfa→kafka does not resolve. Fuzzy matching is effective on descriptive tags and close to inert on very short ones, and that same property is what keeps unrelated short words apart.

Open-vocabulary entity labels. Entity labels could classify a fact against a fixed vocabulary, or capture one free string. Neither covers the case where you want several values that can't be enumerated when the bank is configured — the names a thing is known by, including abbreviations and alternative spellings; the tickets a fact cites; product codes.

The new multi-text type is a list of strings with no fixed vocabulary, so the extractor writes as many values as the content warrants. Each value becomes its own entity and, with tag: true, a tag — so a bank can derive a classification from its own content and then filter on it at recall, without the caller ever supplying the vocabulary. Both client wrappers can express this now; the TypeScript one previously had no way to configure entity labels at all.

Take the synthesized layer with you

A document transfer moved documents and observations — the raw layer. Everything synthesized on top of it, the knowledge pages and mental models a bank had spent real LLM budget building, stayed behind and had to be rebuilt from scratch on the other side.

Pass include_knowledge_base on a whole-bank export (or tick the box in the control plane) and the archive carries the knowledge pages and mental models too, alongside the documents. It's opt-in and off by default, and only available when exporting a whole bank rather than a document list, because a page synthesized from the whole bank means nothing next to an arbitrary subset of it. The manifest reports what came along, so an archive tells you whether it holds a knowledge base before you import it.

The import side preserves what the export captured: mental-model content survives the round trip intact, and observations keep their freshness rather than arriving as though they'd all been recorded at import time.

Every bank on one page

The control plane used to open on a "Welcome to Hindsight" placeholder, so surveying a server that runs several banks meant switching into each one. It now opens on a list of every bank, with its name, memory count, last activity and creation date, paged like the other tables.

Delete in bulk. Select rows and delete them together instead of visiting each bank's danger zone. One failed delete doesn't stop the rest: the banks that couldn't be deleted stay listed with their error, still selected, ready to retry.

The header logo now links back to this page, and the bank selector pins the bank you're in to the top of the list. A server with no banks at all still shows the welcome card, now with a button to create one.

Also in this release

• A registry for extensions shipped outside the server. hindsight-extensions/ is now where extensions distributed separately live, with the Supabase tenant extension as its first entry (hindsight-ext-supabase-tenant) — which takes a third-party IdP's JWT client out of every Hindsight install's dependency graph. If you use it, this is an upgrade step: the built-in path hindsight_api.extensions.builtin.supabase_tenant no longer exists, so an install still pointing at it fails at startup with ModuleNotFoundError. Add the extension to your image and set HINDSIGHT_API_TENANT_EXTENSION=hindsight_ext_supabase_tenant:SupabaseTenantExtension; every HINDSIGHT_API_TENANT_* setting and the schema naming are unchanged. New alongside it: StaticKeysTenantExtension, env-configured per-user API keys with per-schema isolation, for deployments that want multi-user isolation without an identity provider at all.
• Meta Model API as a first-class LLM provider. HINDSIGHT_API_LLM_PROVIDER=meta routes extraction, reflection and consolidation to the Muse family. Note that Muse Spark always reasons — reasoning_effort: "none" is rejected, and reasoning tokens bill against the output budget, so give the per-operation token limits headroom.
• Reflect stops inventing numbers for gaps. Asked about a period the bank holds no data for, reflect could extrapolate a specific figure from neighbouring periods and present it as reliably inferred. It now says the value isn't recorded, while still drawing qualitative conclusions from what is. The same release stops mental-model and knowledge-page delta refreshes from overwriting stored content with what the newest batch alone says — a running count replaced by the batch's count, or an earlier recorded event erased because the batch didn't mention it.
• Bulk ingest no longer starves other banks. The worker rotates its slots across banks, so one bank ingesting a large corpus can't hold every slot while everyone else waits.
• Listing mental models returns metadata by default. GET .../mental-models defaulted to every model's full synthesized content, which bloats a caller's context and pulls a bank's whole synthesized knowledge in one call. It now defaults to detail=metadata; pass detail=content or detail=full to get content back. The MCP list_mental_models tool is metadata-only — read a specific model with get_mental_model.
• A Business Executive bank template, Prometheus ServiceMonitor support and extraContainers/extraInitContainers in the Helm chart, and optional CPU profiling to the logs, controlled by environment variables.

Breaking changes

The bank profile and background endpoints are gone. GET/PUT /v1/default/banks/{bank_id}/profile and POST /v1/default/banks/{bank_id}/background have been deprecated for several releases. They now answer 410 Gone with the replacement call in the detail. The routes stay in the OpenAPI spec with unchanged signatures, so no generated SDK method disappears from under a caller — only the behaviour changes.

Disposition traits and the reflect mission are bank configuration, and already were: read and write them through the bank config API. The display name these endpoints also returned is on the bank list. To make the config API a complete replacement, reading GET .../config is no longer gated on HINDSIGHT_API_ENABLE_BANK_CONFIG_API — that flag now gates only writes, because a bank must always be able to read its own resolved settings.

The control plane and the CLI moved in the same change. One behaviour difference worth knowing if you scripted against it: hindsight bank background now replaces the mission rather than LLM-merging into it, and warns that it does.

curl is no longer in the Docker images. It had one in-image consumer — the standalone image's own readiness loop, now replaced — and it was the sole reason nine unfixable HIGH vulnerability findings shipped in every image. Hindsight's images define no HEALTHCHECK, so nothing breaks by default, but if your Compose healthcheck or a Kubernetes exec probe runs curl inside the Hindsight container, it will now fail with "command not found". Switch it to an HTTP probe (Kubernetes httpGet against /health/live) or run the check from outside the container.

Overloaded servers refuse instead of queueing. Recall, reflect and retain now pass through admission control, on by default. When a worker is at capacity, a request waits for a slot up to a deadline — 30 s for recall, 5 s for reflect, 2 s for retain — and is then refused with 503 and a Retry-After header. Before, it queued without limit behind the concurrency cap, and under a burst the latency simply moved into that queue. The Python and TypeScript clients retry recall and reflect on 503, honouring Retry-After. They don't retry retain, because a retried synchronous retain could write twice. So if you run synchronous retain in bursts, handle 503 — back off and retry, or switch to async retain, which returns as soon as the operation is queued and isn't subject to this limit. The limits are derived per worker from the CPU the process has; tune them with HINDSIGHT_API_ADMISSION_{RECALL,REFLECT,RETAIN}_MAX_IN_FLIGHT and _MAX_WAIT_MS, and set a lane's MAX_IN_FLIGHT to a negative number to turn it off.

The count_chunks hook is gone from the memory extension API. Nothing called it. If a custom memory-store extension implements it, the method is now dead code and can be deleted.

---

0.10.0 carries a long list of fixes across retain, recall, embeddings, the CLI and the supported providers. See the changelog for the full list.