Story How it works Tests Insights Whitepaper What's next Download Author Contact
← Back to Insights
Technical

How I reduced my AI context from 30K tokens to 3.5K without losing precision.

Published May 11, 2026

Pasting context into every session doesn't scale

Every AI conversation needs context: your projects, your decisions, your preferences. Without it, the AI starts from zero every time.

The usual approach is to paste a context file at the start of each session. The trouble is that context files grow. As you add projects and detail, you're pasting thousands of tokens into every conversation. Much of that uses up the context window on information the AI doesn't need for the task at hand.

I was maintaining dozens of knowledge files across 4 businesses and pasting them by hand into every conversation, watching tokens go to redundant context.

Compiling many files into one BRAIN.md

Instead of pasting raw source files, I built a compiler that reads all my knowledge files and produces a single optimized output.

I write in markdown, with one file per topic: one project, one file; one tool, one file. The structure is flat, with no folders. A Python compiler (1,173 lines, zero external dependencies) then validates the format, deduplicates overlapping content, fixes broken references, strips unnecessary noise and generates a single BRAIN.md. Any AI reads that one file, which is always current and verified with a SHA256 checksum.

From 114K characters to 3.5K tokens

From ~114K characters of scattered markdown, the compiler produces ~3.5K tokens of compiled output. That's roughly a 30:1 reduction with near-zero precision loss: no hallucinated summaries and no dropped data.

It gets there in three ways. First, each neuron starts with a brain: summary line (marked with an arrow in the source). The compiler uses that line in the output and adds a pointer to the full file for when more depth is needed. Second, tool and agent definitions are compressed to single-line entries with pointers to the detail. Third, sensitive patterns (API keys, file paths, personal IDs) are redacted from the compiled output automatically.

Recompiled in 8 seconds, without my help

The AI now reads ~3.5K tokens instead of the full source material. It gets the same information, shaped to fit its context window.

I don't have to do anything to keep it current. A file watcher detects changes and recompiles in about 8 seconds. To try it on your own notes, start with a few files and a summary line at the top of each: that line is what ends up in BRAIN.md.

Javier

One compiled BRAIN.md instead of dozens of pasted files.


EIDARA is the open-source project behind this compiler. You can find it on GitHub and on the website.

Keep reading

I stopped organizing my AI's memory into folders How I tested an AI system with 4 different models