Using Microsoft GraphRAG with OpenClaw for persistent AI agent memory
AI agents forget everything. Every session, every time, no matter how capable the model. That’s not a model problem. It’s an architecture problem, and most people are solving it with the wrong tool.
I’ve been wiring Microsoft GraphRAG into my OpenClaw agent as a persistent memory layer, and the difference in how the agent actually reasons is significant enough that I wanted to write it up properly.
The Limits of Stuffing Context
The first instinct when an agent needs memory is to throw documents into the system prompt. It works, up to a point. A handful of reference files, some recent session notes, fine. But the moment you’re dealing with months of project history, dozens of decision logs, and reference material that spans multiple workstreams, the context window becomes a liability. You’re not reasoning anymore. You’re skimming.
Standard RAG is the next step up, and it’s genuinely useful. Convert your documents to vectors, do similarity search at query time, retrieve the closest chunks. Ask “what did I decide about the authentication flow?” and you’ll probably get a useful answer. It’s fast, it’s cheap, and it handles a lot of real-world cases well.
But it fails the moment the question requires synthesis across time. “What dependencies have accumulated across this project over the last three months?” is not a similarity search question. There’s no single chunk that answers it.
What GraphRAG Actually Does Differently
GraphRAG, the open-source system from Microsoft Research, takes a different approach. Instead of indexing documents as vectors, it reads everything at index time and builds a knowledge graph. Entities, relationships, community clusters. The cross-document reasoning happens during indexing, not at query time. By the time you ask a question, the structure already exists. You’re traversing a graph, not hunting for similar text.
That distinction matters in practice. I pointed GraphRAG at a folder of session notes, project logs, and reference documents. Initial indexing for around 55 documents took about 25 minutes using gpt-4o-mini for extraction, which is the right model for bulk processing at that scale. Queries run on gpt-4o for better reasoning on the output side.
The agent setup is straightforward. GraphRAG runs a local REST server. Before the agent admits it doesn’t know something, it queries that endpoint with a simple GET request. The agent gets a synthesized answer built from the graph rather than a raw chunk dump.
What It Actually Found
The test that convinced me was a broad synthesis question about a project spanning several months of documents. No conversational context, just the files. GraphRAG connected threads from documents written months apart, surfaced dependencies that weren’t explicit in any single file, and identified decisions that were only visible by looking across the whole corpus.
A vector search would have returned relevant-ish chunks. The graph traversal returned the actual shape of the problem.
That’s the distinction that’s hard to explain until you see it. Vector search finds documents that mention similar things. Graph search finds how things are connected across documents.
Where It Still Falls Short
I want to be honest about the rough edges because they’re real.
GraphRAG doesn’t do temporal reasoning natively. If an older document contradicts a newer one, it surfaces both and doesn’t know which is current. For a memory system, that’s a meaningful limitation. You have to handle recency logic yourself.
Query latency runs around 20 seconds. That’s fine for a background agent that’s doing research or synthesis before responding. It’s not fine for anything interactive. This is a tool for agents that think before they speak, not for real-time chat.
The incremental update mode (graphrag update –method fast-update) works well for daily maintenance without rebuilding the full index. I run it on a cron at 2am. New documents get added without touching what’s already indexed. That’s the piece that makes the daily cadence practical.
Why This Architecture Makes Sense
The agent memory problem doesn’t get solved by better models. A longer context window helps at the margins but doesn’t change the fundamental issue, which is that unstructured document dumps don’t scale and don’t reason. A knowledge graph built from your actual history is a different kind of memory, closer to how a human expert actually retains institutional knowledge than a pile of text files is.
The setup is not trivial. Initial indexing time, latency on queries, and the temporal reasoning gap are all real costs. But for any agent that needs to reason across months of real work rather than just retrieve recent facts, the tradeoff is worth it. The agent I’m running now actually knows the history of the projects it’s working on. That’s not a small thing.
Sources & Further Reading
#AIEngineering #GraphRAG #AIAgents #OpenClaw #RAG #LLM
