Published June 16, 2026
Six months ago I started building a personal memory system because every note tool I’d tried had the same flaw: it assumed I’d happily feed it. Notion, Obsidian and Roam all begin with “you write, you tag, you link.” If you don’t curate, nothing grows, and curation is where most of these systems die without anyone noticing.
So I inverted the question: what if the memory curated itself? I built a local autonomous curator that digests the facts I drop during the day, folds them into the notes, removes duplicates, flags contradictions and archives what’s old. That was the last two months of work, and I call it the Librarian. It runs around the clock, every few minutes. Last week it went well past a thousand autonomous cycles with zero errors and several hundred facts integrated.
But the curator’s appetite was always bigger than my willingness to feed it. Every fact I wanted remembered, I still had to type. The whole point was to remove the friction between “I learned something” and “the memory has it,” and the friction had only moved to the input.
Step 4 of the roadmap, finished today, was senses. I mean that literally, not in a sci-fi way: passive capture of what already goes through my hands, ears and eyes on a normal day. There are three modes and one rule.
The first is voice. I hold down a keyboard shortcut, speak and release. A local speech model transcribes in a second or two and pastes the text wherever my cursor is. There’s no assistant, no floating window, no “summon the AI.” It’s an input device with no opinion about what I just said.
The second is typing. Nothing I type is kept while I’m typing it. When I press Enter, and only then, what I sent goes to a local model that decides whether there’s anything worth remembering. Most of the time the answer is no: I just sent a message, ran a search or typed a command. When it’s yes, the fact enters the curator’s pipeline like any other source. The rule is the commit: the system only learns what I confirm with Enter. If I delete before Enter, nothing is kept. It’s a small choice that does a lot of work for privacy.
The third is the screen. When I stay on a window for a while, the memory reads the text it shows, the way screen readers for blind users do. It doesn’t look at pixels; that would be OCR, which is fragile and a little creepy. It gets what the app itself reports it is showing. A local model reads that text and tries to pull out a single substantive topic, with a strong bias toward “nothing here.” If there is something real, the memory notes that I was exposed to it today.
A system that listens, reads and watches cannot be passive about privacy. That’s why the most important new feature is the off switch rather than any of the three senses.
There’s a small icon in the system tray with three modes. Low is the default: the memory learns from everything I type, dictate and stay on. Medium suspends voice and screen, so only what I type with intent gets through. High suspends everything. It takes one click, with no restart, no menus and no “are you sure?” The listening stops.
Underneath all three modes there’s a floor that never moves: some places are off-limits to the senses in every mode, and that can’t be tuned.
Today is day one of a seven-day pilot: a week of real use, after which I’ll check whether the system learned things I would otherwise have forgotten without noticing. If it did, Step 4 ships. If not, I tune the prompts and start over.
Two hours in, I haven’t typed a single fact into the memory by hand today, and it has kept growing in the background while I worked. The bottleneck has moved upstream, from “I’d remember to type this” to “the system might learn this if I spend enough time on it.”
The next post is about the third sense in particular: why I turned down the obvious option, computer vision, for a much stranger one. The design reasons deserve their own page.
The pilot ended and the senses shipped. After two weeks of daily use, the visible difference is that I write notes less often and just work. At the end of the day the system has captured more than I would have by hand, and most of it lands in the right topic.
Voice changed capture the most. It went from “open a note, type the thing” to “say the thing,” and that changes how often I capture, which changes how much the system knows. Typing picks up the facts I would have written down anyway: a name, a price, a deadline or a decision mentioned in a chat. The screen sense keeps the topic, not the content, so the memory knows I spent an hour on a project or researched a vendor without keeping what was on the page.
The tray modes found their rhythm too. Most of the time I leave the relaxed mode on. When I work on something sensitive, one click pauses everything, and the intermediate mode keeps typing on while the screen pauses.
The less visible difference matters more to me. EIDARA used to be a system for what I told it; now it’s a system for what’s going on around me. That is closer to how memory forms in a person, through passive observation more than deliberate capture.
The senses aren’t perfect. They miss things, and they sometimes save things that aren’t useful; the cleanup layer takes care of most of that. What I still need to measure is how much of that first pass I end up discarding, and whether the share goes down as the cleanup improves.
Javier
Voice, typing and screen capture, with an off switch in the tray, are part of what I’m building after EIDARA v2.