Open Agentic Brain and Memory
What's the difference between long context, RAG and memory?
They differ in what reaches the model, and who decides. Long context sends everything. RAG fetches the relevant parts of a fixed document set. Memory fetches the relevant parts of what the agent itself learned in earlier sessions.
| Long context | RAG | Long-term memory | |
|---|---|---|---|
| What the model reads | Everything | Top matching documents | Top matching memories |
| Source | Whatever you paste | A fixed corpus | Written by the agent as it works |
| Survives the session? | No | The corpus does | Yes, and it updates |
| Cost per call | Grows with history | Small, flat | Small, flat |
| Hard limit | The window size | None | None |
In practice all three meet in the same place: the model's context window. Retrieval and memory are ways of deciding what earns a place there. For the bigger picture, see our guide to context engineering.
When everything fits, does long context win?
Yes, on accuracy, by a small margin. If the whole history fits in the window and you only ask once, giving the model all of it is the ceiling to beat.
We tested this on LoCoMo, a public benchmark of 1,540 questions about long conversations. Each conversation is about 20.8 thousand tokens, small enough for any modern model. The same model answered and judged every run.
LoCoMo, categories 1 to 4, 1,540 questions, LLM-judge accuracy. Same model answering and judging in every row.
The whole conversation wins, and that matches published research: a widely cited 2025 study also found full context ahead of every memory product it tested. Retrieving more, and re-ranking what was retrieved, closes most of the gap. Questions that need several parts of the conversation improved the most (from 65.6 to 80.5), and on open-ended questions the two were tied: one point apart on 96 questions, which is about one question.
What does long context actually cost?
You pay for every token in the window, on every call. The same 1,540 questions:
| Accuracy | Tokens per question | Cost, all questions | |
|---|---|---|---|
| Whole conversation | 89.3 | ~20,800 | $3.36 |
| Memory, 40 results | 86.6 | ~3,700 | $0.73 |
| Memory, 20 results | 78.8 | ~2,000 | $0.47 |
So full context bought about 3 points of accuracy for about 4.6 times the price. Whether that's worth it depends on how often you ask. For a one-off question, maybe. For a coding agent that starts dozens of sessions a day, each reading the same growing history, the difference compounds into most of the bill. Larger windows are also slower: more input means a longer wait before the first word of the answer.
What happens when the history doesn't fit?
Then there is no full-context option, and retrieval is the whole game.
LongMemEval-M puts about 1.27 million tokens of history, roughly 2,500 conversation rounds, behind every question. That's beyond the window of any model you can buy today. Memory still answered 83.0% of its 500 questions correctly, reading about 19 thousand tokens per question, 98.5% less than the history.
This is the situation real agents end up in. A few weeks of daily coding sessions is already more history than a window holds, and it never stops growing. A ten-times larger history cost the memory setup 7 points (90.0 on LongMemEval-S, 83.0 on M), not everything.
Why do models slip on long input?
Because attention is spread across everything you send, and not evenly.
Lost in the middle
Facts buried in the middle of a long input are found less reliably than facts near the start or end.
Context rot
As input grows long and noisy, irrelevant or outdated material competes with what matters.
Stale facts
A full history contains every old decision too. The model has to work out which one is current.
Distractors
Similar-looking but wrong passages are more likely to be picked the more of them there are.
Full context still came out ahead on LoCoMo, so these effects don't erase its advantage when the input is modest. They do grow with length, which is one more reason retrieval holds up better as history grows.
Is RAG dead now that context windows are a million tokens?
No. A bigger window raises the limit; it doesn't decide what belongs inside it.
- Cost and speed scale with what you send, every time you send it.
- History keeps growing. Any fixed window is eventually too small for an agent that works every day.
- Nothing persists. A window is empty at the start of each session unless something refills it.
- Accuracy on long, noisy input is lower than on a short, relevant one.
What has changed is the shape of retrieval. Fetching a handful of loosely matching chunks isn't enough; the memory runs above retrieved 40 results and re-ranked them, and that's what closed most of the gap to full context.
How do you choose between long context, RAG and memory?
Ask four questions.
- Does it fit comfortably? If the material is well within the window and you ask once, send all of it.
- Will you ask again? Repeated questions over the same material pay the full price every time. Retrieve instead.
- Is it a fixed set of documents, or knowledge built up over many sessions? Documents suit RAG; decisions, preferences and lessons suit memory.
- Will it outgrow the window? Agent histories always do. Plan for retrieval before you hit the wall, not after.
What's the hybrid that works best?
Recent work in full, older work through memory.
Keep the current conversation and the files you're working on in the window, where full context is strongest. Let memory carry everything older: the decisions, preferences and fixes from past sessions, retrieved when the current task needs them and kept small. The window stays short and relevant, and nothing important is lost when the session ends. For how such a layer works, see our guide to AI memory layers.
Frequently asked questions
Is long context better than RAG?
When everything fits, sending all of it is usually the most accurate option, but it costs more on every call. Retrieval gets close for a fraction of the tokens, and it's the only option once the material no longer fits.
Is RAG dead now that context windows are a million tokens?
No. Bigger windows don't remove cost, latency or the accuracy loss on long, noisy input, and agent histories keep growing past any fixed window.
What's the difference between RAG and long-term memory?
RAG retrieves from a fixed set of documents. Long-term memory is written by the agent as it works (decisions, preferences, lessons), updated when they change, and retrieved when a later task needs them.
How much does long context cost compared with retrieval?
On LoCoMo, sending the whole conversation cost about 4.6 times as much as sending retrieved memories, for about 3 points more accuracy.
When should I use full context instead of retrieval?
For one-off questions over material that fits comfortably, when accuracy matters more than cost. For repeated questions, growing histories or anything larger than the window, use retrieval or memory.
Numbers measured by us on public benchmarks (LoCoMo; LongMemEval S and M), October 2026, with the same model answering and judging within each comparison. Costs are at the API list prices on the day of the run.
Keep the window small. Keep everything else.
Open Agentic Brain and Memory is an open-source, self-hosted memory layer for AI coding tools. It remembers what matters across sessions and brings back only what's relevant, on your machine and under your key.
curl -fsSL https://raw.githubusercontent.com/rohithhn/Open-Agentic-Brain-and-Memory/main/install.sh | bash