Published July 16, 2026
A few days ago I wrote about the moment a second AI vendor connected to the same local memory: OpenAI’s Codex, reading and writing the same brain as Anthropic’s Claude. Now a third has joined. DeepSeek is also working with EIDARA.
So today three frontier models from three different companies (Claude, ChatGPT and DeepSeek) read the same compiled memory and contribute to it. That memory is about 630 notes, compiled into a single file of roughly 18,500 tokens that each assistant reads at the start of a session.
The third connection is not what this post is about, though. It’s about what happens when you ask three independent models, trained by three different labs, to look at the same system separately and tell you what’s wrong with it.
They don’t audit the same way, and that difference is where the value comes from.
Claude brings continuity. It’s the model this system was built with over the past year, so it knows where the wires are and which design choices are deliberate and which are scar tissue. Its job now is less about inventing new ideas and more about processing the other two’s proposals and deciding what ships.
ChatGPT brought a security lens. Its sharpest contribution was about permissions, and it was the only one of the three to go at the point head-on. The others assumed that if a client’s access is written down in a config, that’s the boundary. ChatGPT’s objection was blunter: I can’t verify that boundary from where I’m standing, so it isn’t one. A written rule the client can’t check is a promise, not a wall. That single observation turned into a batch of work on how the memory server proves who a client is before trusting it.
DeepSeek brought two things nobody else did. The first was efficiency: concrete proposals to do routine work with a lighter local model and to cache repeated decisions, which cut routine model calls by roughly 40 to 50% with no measurable loss in quality. The second is the one I keep thinking about. It was the only model that treated the local curator as a single point of failure. The others assumed it would keep running; DeepSeek assumed it would fall over one day and asked what the plan was. There wasn’t one. There is now.
With a single reviewer, even a very good one, you can never fully separate its insight from its blind spots. A model trained one way tends to notice one kind of problem and miss another.
When three models from three different labs read the same system on their own and land on the same diagnosis, that is about as close to an objective signal as I can get. It stops being one house’s opinion and becomes a problem real enough that three different kinds of model all tripped over it.
They converged more than I expected. All three, without coordinating, said the system’s watchdog should act on problems rather than only report them. All three said the memory handed to an assistant at the start of a session had grown too heavy, with too much useful history sitting in a “cold” tier the assistant never saw. And all three said the protocol for how models write into the memory needed formal versioning, so that a mismatched client can’t corrupt anything.
Those convergences went to the top of the list. When only one model saw something, I treated it as a lead worth chasing. The ChatGPT permission catch and the DeepSeek resilience catch both became real work precisely because they were blind spots for the other two.
By cross-referencing the three audit lists and building from the overlap, I closed close to thirty structural changes in the first week of the trio. They were refinements rather than new features, and that distinction defines this phase.
A few of them, in plain terms. The memory a session starts with is now delivered in layers, so an assistant gets a light overview first and pulls detail on demand instead of swallowing everything up front. The background curator now pushes back when it’s overloaded instead of falling behind without a word. The watchdog opens its own tickets when it finds a problem. And the write protocol between a big model and the memory is now versioned, with conformance checks that block a client whose version doesn’t match. In the observation window, with three models writing to the same brain, I’ve logged zero cross-vendor contradictions, so the write rules are holding in practice.
One caveat matters for the phase I’m in. The maintenance backlog those changes were meant to drain isn’t flat yet: it dropped below target for days, then spiked again under a wave of new input. Calling it “stable” today would inflate the result. It’s better, not done.
One number, with its limits stated: on a fixed set of 31 hard retrieval questions that I run continuously as an internal check, top-answer accuracy went from 6.5% to 25.8% over this window, nearly four times the baseline. That is my own benchmark, not an industry one, which brings me to the point of this post.
For most of the past year the question was what should this system do? With the three models’ input processed and the list drained, the question becomes how good is it next to everyone else?
So the next phase is benchmarks, the public ones the field uses: LoCoMo, LongMemEval and BEAM. Until now EIDARA has been optimized for the everyday life of one specific user. Benchmarks are the cold comparison against the other systems in this space (Mem0, Zep, Letta, Hindsight) under conditions I didn’t design.
To be plain about where I stand: EIDARA has no public benchmark score today. The first step is to publish my internal benchmark with the full method and the exact questions, so anyone can see how I measure before I claim anything, rather than lead with a headline number. Then I’ll run the public benchmarks and publish what comes back, good or bad. I asked all three models for a second list, this one only about benchmarking, and I’ll cross-reference those lists the same way to separate what is worth measuring from each lab’s house style.
I’m not trying to win a ranking. I want comparable numbers I can stand behind before EIDARA goes further out into the world.
Three models from three vendors now read and improve the same brain in daily use, and none of them owns it. That part of the design works; what’s left is measurement. The internal benchmark with its method and questions goes out first, then the LoCoMo, LongMemEval and BEAM runs, and I’ll publish those results here whatever they are.
Javier
Three models from three vendors on one memory is part of what I’m building after EIDARA v2.