At the end of August I wrote about a personal project I’d been building.
It was a memory system for Claude Code. Something that worked differently from most memory systems. I was trying to capture the memory of what I interacted with, how, and why. Not notes in a shared knowledge base. The record of how I interacted with that knowledge base.
I call it rutter.
It’s built on a simple idea. Summaries aren’t records. If you keep consolidating and updating what the AI knows, you lose the history. You can’t go back and work out “how did I get here?”
I built it so that my Claude Code sessions built the record unobtrusively while I worked. It’s helpful. I can always ask “what was I working on the last few days?” and Claude Code will point me in the right direction.
But more recently - I think like many of us - I’ve started using a wider range of models and tools.
Whose memory is it anyway?
AI tools are fairly good at some kinds of memory. ChatGPT has memory. Claude has memory. Coding agents use mechanisms to retain context, write handoff notes, etc.
They’re all separate.
I often spend a morning exploring a plan with Claude Code. Eventually we reach a point when it’s time to write up a plan, or some early pieces of functionality. A couple of days later, I come back to it. But either through circumstance or preference, I want to take a look at it with a new model.
Any of them - Codex, GitHub Copilot, Antigravity - can read the repository. Inspect my code, inspect my plans. Establish a good amount of what I’ve built or planned.
But it doesn’t mean it knows anything about how we reached that decision last Tuesday. Did we reject alternatives? Which notes are the most pertinent to the decision we made.
I could explain it all again. Although I’ve probably forgotten some details myself.
I could put everything into handoff notes. A document for the next tool to read it. I can copy my session summaries from one tool to another.
Those are all reasonable solutions to the problem. They’re also all more work for me.
But the code is mine. The plan is mine. The notes are mine. The decisions were mine.
So this is a weird arrangement.
Why does the memory of how we made them depend on the AI tool that happened to be open that day?
One record, multiple agents
So that’s the direction I’ve taken rutter, to try and solve that problem.
The first version had decisions from Claude Code sessions. It recorded a line about what happened, and references to those notes.
I needed to extend that to other clients.
Today, rutter supports capture from Claude Code, Codex, Grok, GitHub Copilot CLI, Antigravity CLI, and Muse Code.
They don’t all work in exactly the same way. Some of them can get at the session transcript. Others just see a final reply. So my capture hooks have to work with what each client makes available to me.
But I’m putting all the records in one place.
I have a folder of Markdown files that sits alongside all my other notes. A local SQLite index is there to make them searchable. And every configured client can reach the records through MCP.
I can make that decision in Claude Code. Next week I can ask Codex about it.
I can start something in Muse. In a few days I can ask Grok, “hey what was that thing I was working on?”
It doesn’t need to synchronize. There’s no service or model. It’s a shared common record.
Its main value is that the memory exists independently of the tools that use it.
If Codex went away tomorrow, I wouldn’t lose the history of the work I’d been doing with it.
What survives a handoff?
As an example, imagine I’m working on a newsletter or a blog. I’m trying to build a publishing schedule.
I look at audience research. I think about how often I could publish decent quality content. I decide to change from weekly to fortnightly publication.
My session records that decision, with references to the notes involved. I don’t have to ask it.
I’m really busy, so I don’t get back to this project for another couple of weeks. I fire up a different one of my AI clients and ask “hey, what was that decision I made about the newsletter recently?”
The second client can retrieve the original decision. When it was recorded. Which notes were referenced when we made it.
Now it has a record to work from, without having to reconstruct an explanation only from what’s in the repository today.
That’ll work even if I reorganize my notes. If I’ve moved a file but not changed the content, rutter will find it again.
If it can’t find the reference, then it tells me. It doesn’t just pick something and hope it’s plausible.
AI doesn’t like empty space. There’s a temptation to make recall feel frictionless. So the machine should have an answer that makes it seems like it remembers.
But I’d prefer that the record stop, once it’s told me all it knows.
It’s not perfect. Even if it captures a content hash after the session, I can’t prove the model actually read any particular note. I can’t always tell if it relied on that note to help it reach a conclusion.
I want to keep these limitations visible.
Different continuities
Even more recently, I’ve tried to extend rutter beyond just session records.
I use it today to answer a straightforward kind of question:
What was I working on?
The second is a bit more complex:
What did I think about this, and has that thinking changed?
In one working session I might decide on a particular approach. But a week or two later I find some new evidence. It changes my mind.
Memory systems will, by and large, record and remember the newer position. That’s sensible - we generally want a machine to work from current beliefs and stances.
But those memory systems also tend to destroy the earlier position. And that can destroy something useful.
When did I change my mind? Why? Or might I want to revisit an approach I abandoned before, because the circumstances have changed now.
rutter has its own record of those positions. Agents can record whether they’re asserted, revisted, reaffirmed, or retired.
They’re still there, and in order.
I can ask rutter for the current position, it’ll give me the latest recorded stance. If I ask it for its history, it can show me the chain of changes.
It’s still quite simple, and that’s on purpose. The client will decide what qualifies as a decision that it’s worth recording. The current position can be reconstructed from all those recorded events. And we can reindex any new position records so they’ll appear in the recall.
I’m trying to avoid creating a system that keeps repairing the past, so that it’s compatible with the present.
If the system is continuously rewriting its understanding, I can’t tell between what I actually decided, and what the system thinks I decided.
I want to map how I got here.
Memory is infrastructure
The most useful infrastructure tends to seem unremarkable.
We don’t build a design system for the sake of having a design system. We actually want to build it so that people aren’t thinking about it when they build a screen. They just know that the decisions they need have already been made. The consistency, accessibly, and behavior are dealt with. They can focus on creating delightful experiences within those boundaries.
Memory should work the same way.
I was wasting time at the start of new sessions trying to reconstruct what I’d done in the last one.
I shouldn’t have to do that. Not even if I’ve changed tools. And I don’t think I’m alone in working more and more with multiple agents.
My work moves between them.
If every agent’s version of work history is different, that’s going to be a lot of time wasted on reintroduction.
Of course, not all memory is the same. Personal preferences, project history, organizational knowledge, and records of decisions are different things. They have different requirements around ownership, reliability, and access.
Rutter isn’t trying to solve those problems at an organizational scale. It’s supposed to be one person’s tool to share records across their tools. Not multiple people trying to put together a common history.
I’m still experimenting
When I published the original article, I’d been using rutter for about ten days. It had recorded 193 session entries and references to 106 different notes.
Those entries and notes continue to grow. And I find it fairly natural to refer to them during my sessions; whether through direct invocation or because the models automatically use rutter.
The big question is whether the memory is useful. Does it stop me repeating work. Can it recover an old decision that I’d otherwise have forgotten? And does tracing back that earlier position actually make any difference to me today?
Adding support for more agents doesn’t resolve that question.
So I’ll keep testing.
The code is open source, under MIT license.
My original idea was that a store of knowledge isn’t the same as the memory of how we used it.
I feel confident that’s right.
The second part is what I’m adding now.
The records of what I did should remain mine, whichever agent helps me work with them next.
Further reading:
A summary isn’t a record. So I built a librarian. - the original project outline.
He, I. Context engineering: memory, compaction, and tool clearing. Claude Cookbook, Mar 2026.
Barber, G. You Can Have Collaborative Software That’s Wary of the Cloud. Wired, Dec 2019.
Memory Models: Towards Agents That Learn. Letta Blog, Jun 2026.