Your CrewAI Agents Forget Everything Between Runs. Here's the Fix.
CrewAI agents lose all memory when a crew finishes. hindsight-crewai plugs into CrewAI's ExternalMemory to persist knowledge across runs -- three lines of setup, and your agents automatically store task outputs and recall relevant context.
The Problem: Stateless Crews
CrewAI has a memory system. Short-term, long-term, entity memory. It works well within a single kickoff().
Then the process exits.
Next run, the crew starts from zero. Every fact learned, every decision made, every entity discovered -- gone.
This matters when you build crews that run repeatedly:
- A research crew that deepens knowledge over time
- A support crew that remembers customer history
- A planning crew that tracks decisions across sprints
CrewAI's built-in memory backends (RAG storage, SQLite) are designed for single-run persistence. For cross-run, cross-session memory that actually compounds, you need something else.
That's what the hindsight-crewai package does. It implements CrewAI's Storage interface using Hindsight's memory engine, so your crews remember everything -- across runs, across days, across weeks.
Architecture
Here's how it fits together:
CrewAI Crew
└─ ExternalMemory
└─ HindsightStorage (implements Storage interface)
├─ save() → Hindsight retain (extract facts, entities, relationships)
├─ search() → Hindsight recall (semantic + graph + temporal retrieval)
└─ reset() → Hindsight delete_bank + recreate
CrewAI calls save() after each task completes and search() before each task starts. You don't manage the lifecycle -- CrewAI drives it, Hindsight stores it.
Under the hood, Hindsight does more than store text. It extracts structured facts, identifies entities, builds a knowledge graph, and runs multi-strategy retrieval (semantic search, BM25, graph traversal, temporal ranking) with cross-encoder reranking.
Your crew gets a real memory system, not a vector dump.
Step 1 -- Start Hindsight
Install and start the memory server:
pip install hindsight-all
export HINDSIGHT_API_LLM_API_KEY=YOUR_OPENAI_KEY
hindsight-api
This runs locally at http://localhost:8888 with embedded Postgres, embeddings, and reranking. No external infra needed.
Note: You can also use Hindsight Cloud and skip the self-hosted setup entirely.
Step 2 -- Install the Integration
pip install hindsight-crewai
This pulls in hindsight-client and crewai as dependencies.
Step 3 -- Wire It Up
from hindsight_crewai import configure, HindsightStorage
from crewai.memory.external.external_memory import ExternalMemory
from crewai import Agent, Crew, Task
# Point at your Hindsight instance
configure(hindsight_api_url="http://localhost:8888")
# Create agents
researcher = Agent(
role="Researcher",
goal="Find accurate, detailed information on the given topic.",
backstory="You are a thorough researcher who digs deep into topics.",
llm="openai/gpt-4o-mini",
)
writer = Agent(
role="Writer",
goal="Write clear, well-structured content based on research.",
backstory="You are a technical writer who values clarity and precision.",
llm="openai/gpt-4o-mini",
)
# Create a task
research_task = Task(
description="Research the benefits of Rust for CLI tools.",
expected_output="A detailed summary of Rust's strengths for CLI development.",
agent=researcher,
)
write_task = Task(
description="Write a short article based on the research.",
expected_output="A polished 3-paragraph article.",
agent=writer,
)
# Create the crew with persistent memory
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, write_task],
external_memory=ExternalMemory(
storage=HindsightStorage(
bank_id="research-crew",
mission="Track research findings, technical comparisons, and writing preferences.",
)
),
)
crew.kickoff()
That's it. After kickoff(), every task output is retained in Hindsight. Next time you run this crew, it recalls relevant prior work before starting each task.
Step 4 -- Run It Again
Second run, different topic:
research_task = Task(
description="Research how Go compares to Rust for CLI tools.",
expected_output="A comparison of Go vs Rust for CLI development.",
agent=researcher,
)
Now the researcher has context from the first run. It knows what it already found about Rust. The writer remembers the style and structure from the previous article.
Third run:
research_task = Task(
description="Which language should I pick for a new CLI tool?",
expected_output="A recommendation based on all prior research.",
agent=researcher,
)
The crew now draws on two prior research sessions. Knowledge compounds.
Step 5 -- Add Reflect for Deeper Synthesis
CrewAI's Storage interface has save and search. But Hindsight also supports reflect -- a synthesis operation that reasons across all relevant memories instead of returning raw facts.
Since reflect doesn't map to the Storage interface, it's exposed as a CrewAI Tool:
from hindsight_crewai import HindsightReflectTool
reflect_tool = HindsightReflectTool(
bank_id="research-crew",
budget="mid",
reflect_context="You are helping a development team evaluate programming languages.",
)
researcher = Agent(
role="Researcher",
goal="Provide deep, synthesized analysis on technical topics.",
backstory="You are a senior researcher. Use the hindsight_reflect tool to review what you already know before starting new research.",
tools=[reflect_tool],
llm="openai/gpt-4o-mini",
)
When the agent calls hindsight_reflect, it gets a synthesized, reasoned response that draws on the full knowledge graph -- not just the top-k vector matches.
Per-Agent Memory Banks
By default, all agents share one bank. If you want each agent to have isolated memory:
storage = HindsightStorage(
bank_id="research-crew",
per_agent_banks=True,
)
The researcher writes to research-crew-researcher, the writer to research-crew-writer. Each agent builds its own knowledge base.
For full control, use a custom resolver:
storage = HindsightStorage(
bank_id="research-crew",
bank_resolver=lambda base, agent: f"{base}-{agent.lower()}" if agent else base,
)
Pitfalls and Edge Cases
1. Bank ID collisions. If multiple unrelated crews share a bank_id, their memories mix. Use unique bank IDs per crew or project.
2. Large task outputs. CrewAI passes the full task output to save(). If your tasks produce very long outputs, Hindsight handles the chunking, but retain latency increases. Set a reasonable expected_output length in your task definitions.
3. Recall budget tuning. The default budget="mid" balances speed and thoroughness. For latency-sensitive crews, use "low". For deep analysis, use "high". Budget affects how many retrieval strategies run and how much reranking happens.
4. Async event loop conflicts. CrewAI runs inside an async event loop. The integration handles this transparently via a dedicated thread pool, but if you're also doing async work in custom tools, avoid calling hindsight-client directly from the same event loop. Use the HindsightStorage and HindsightReflectTool abstractions instead.
Recap
hindsight-crewaigives CrewAI agents persistent, compounding memory- It implements CrewAI's
Storageinterface, so integration is three lines - Memories are automatically stored after tasks and recalled before tasks
HindsightReflectTooladds on-demand synthesis for deeper reasoning- Per-agent banks let you isolate or share knowledge as needed
The integration handles the hard parts: async compatibility, thread safety, fact extraction, multi-strategy retrieval. You just point it at a bank and let your crews learn.
Next Steps
- Try it locally:
pip install hindsight-all hindsight-crewaiand run the example above - Use Hindsight Cloud: Skip self-hosting with a free account
- Add tags for scoped memory: Use
tagson retain andrecall_tagson search to partition memories by project, environment, or topic - Inspect memories in the web UI: Run
hindsight-control-planelocally or use the cloud dashboard to browse facts, entities, and mental models - Combine with per-agent banks: Give specialized agents their own memory while sharing a common bank for cross-agent knowledge