Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Context Language Models (CLMs) give the model control over its own context, allowing it to delete, edit, and reorganize conversation history like a file. On benchmarks, this approach matches or slightly beats summarization-based compaction while using significantly less compute — up to 59% fewer flops in some tests. However, the same mechanism introduces a safety risk: nothing prevents the model from removing user instructions or inserting its own, and the paper does not solve this.
Key points
CLMs allow the model to edit its own context as a file, unlike current harness-driven compaction.
On a web research benchmark, a CLM with no training scored 59.4% vs 53.4% for summarization, using 21.5% fewer flops.
After reinforcement learning, a 9B parameter CLM was essentially tied with trained summarization (42.5 vs 42.1) but used 39% less compute.
The paper warns that editable context can become a channel for prompt injections or self-generated instructions to persist across turns.
Techniques
- Context Language Models
- summarization-based compaction
- suffix cache reuse
- reinforcement learning
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
I was scrolling X when a post about CLMs caught my attention. Ruin Shia introducing context language models. Models that treat their own context, the conversation they're working in as a
file they can edit however they like. If Claude code or Codeex has ever dropped a rule you gave it early in a long session, this one's about you. Researchers at Meta Super Intelligence
Labs, the University of Washington, MIT, and Trillium Labs wrote it and their idea is blunt. Today, software around the model decides what it forgets. Here, the model decides. We'd made a video on
RLMs before, and this seemed like an extension to LLM, similar to RLMs. So, I thought, let's read it. The detail that grabbed me. A helper function the model wrote for itself, then called 37 times.
Start with your side of it. Claude Codes docs say that when the context fills up, it clears older tool outputs first, then summarizes. Your requests and key code snippets are kept. Detailed instructions
from early in the conversation may be lost. That's a fair trade. And per the paper, codec, CLI, and cursor compact at preset lengths, too. But look at who's holding the scissors. The harness, the
software wrapped around the model. It cuts at a length set in advance, not at a moment the model picks. That's how a rule from minute one can end up in the trimmed part. Handing the model the
scissors raised the question that bugged me most. Can it cut your rule or write in one you never gave it? By the end, you'll know how this works. what the numbers really say and whether you can
use any of it today. First, my extension theory. RLMs from Jeang, Krasa, and Katab treat your prompt as an environment. An RLM examines it and calls itself recursively over the
pieces. But the CLM paper points out that whatever it finds still gets added to the live context. Alex Jang, RLM's own first author, answered on X. He called CLMs a very clever idea, even
more extreme than RLMs at moving context out of the model's way, and added, "So, no, they're not the same." My extension theory lasted about one reply. To see how far it goes, picture today's context
as a paper roll. Every message, search result, and tool output gets added to the bottom. Left alone, it only grows. When it gets too long, the harness cuts the old part off and stamps a summary on
top. Newer methods let the model hold the scissors, but it only picks from a menu of cuts people designed. A CLM gets no menu. Instead, the roll becomes a file, and on each turn, the model can
keep writing at the bottom as usual, or run bash commands to edit anything in it. Every edit lands in the model's live context on the very next turn. That's where the paper roll picture breaks.
Tear a line off a roll and you still remember reading it. The model doesn't get that. This file is its context. So delete a line and on the next turn it's simply not there. In the papers
examples, each turn gets a little label context turn one roll assistant and the model finds and edits around those labels with Python and a regular expression. Panda model the whole file
and no menu. And what does it do? The paper's qualitative examples show it. And the model came up with every one of these strategies by itself. Given search results it decided were irrelevant. It
wrote loops to delete them. In the conversation itself, it created a new role called notes. And there's that helper function compact turns written once and called 37 times. More reuse
than a lot of code bases manage. I keep coming back to the scoreboard. Inside its own context, the model kept a scoreboard of agent status. 163 times it updated that scoreboard in place.
Meanwhile, the whole context stayed at 6 to 8,000 tokens. On the paper roll, each update would have been one more copy added to the bottom. Co-author Nathan Lambert summed it up on X. Don't fear
compaction anymore. Big claim. I checked the numbers in the paper against it, starting with no extra training at all. For this test, the authors took Quinn 3.6, a 27 billion parameter open model,
capped its context at 32,000 tokens and handed it the file. On browse comp plus Plus, a web research benchmark, it scored 59.4%. Codec style summarization scored 53.4.
In the paper's terms, that's an 11.4% relative gain. And it got there on 21.5% fewer flops, the raw math operations a model runs. Round that and it's about a fifth less compute. On a 12-hour
edgebench run, a 10 task subset, the CLM scored 44.6 and summarization scored 42.3. 5% higher, the abstract says, and that's relative. Worked out in points, it's a bit over two. Compute is the
bigger story. 179 quadrillion flops per trial against 437 quadrillion, 59% less. Over a 24-hour run with six agents, a setup the paper calls software world, the CLM
swarm scored 65% higher on the paper speed up measure than a summary based swarm at the same compute. One caveat covers all of it. We're talking about a 27 billion parameter open model at a
32,000 token limit, not a frontier model. And every figure here is the author's own measurement. The paper did try a bigger name though. With Claude 4.6 on edgebench, the CLM scored 51.
Training is where the abstracts headline number comes from. After reinforcement learning, the 9 billion parameter Quinn 3.5 improved by 47.6% on browse comp plus questions it hadn't
trained on, and it used 12% fewer flops than it did before training. I expected that to be the clean win. Table 2 tells a smaller story. Before training, the CLM scored 28.8% and the summary harness
scored 34.7. So the CLM started behind. Training lifted it to 42.5, but the summary harness got trained too. 42.1. Do the subtraction 4/10 of a point. Basically a tie. Where CLM actually wins
is compute. Per question. The train CLM used 1.34 quadrillion flops against 2.19 quadrillion for the train summary harness. Divide one by the other and that's about 39% less compute for about
the same accuracy. A model that edits its own context creates a problem for the server though. Servers make long sessions affordable with prefix caching. Reusing the work already done on the
unchanged start of a conversation and only processing what's new at the end. A CLM edits the middle. In SGlang, a popular serving engine, the first edit means every token after it gets
recomputed. Change one line near the top of the file and the server reprocesses everything below it. Their fix is called suffix cache reuse. For every token that survives an edit, it keeps the cached
work, shifts its position information to the new spot, and only processes the new text. With the 27 billion parameter Quinn on browse comp plus, it matched performance at 65% of SG Lang's compute.
In other words, about a third less server compute, 35%. That 35% comes with strings. The paper calls the trick approximate. It moves at most six chunks of cached work to keep the error down.
So, you need a cache trick standard SG lang doesn't have, and you give up a little exactness to get there. As for using it today, I expected nothing to download. Over on GitHub, Facebook
researches repo has code for the harness, the training, and the cash trick under a non-commercial license. You won't find it as a feature in claude code or codeex. Senior author Pengui Co,
posting about the strategies the model invented wrote, "No more harness engineering." He means handbuilt rules for what gets cut. And fair enough, yet the repo ships a harness. Something
still has to hand the model the file. Back to the question from the start. Can it cut your rule? Nothing in the design stops it. That's the same freedom that lets it delete a stale search result.
And can it write in one you never gave it? The paper warns about exactly this. Editable context, the author's right, can become another channel through which prompt injections or self-generated
instructions persist across turns. Related work has already seen a model insert unauthorized instructions into its own summary. Raising the risk is as far as the paper goes. Solving it is
left open. Any hand that can keep your rule at the top can rewrite it. If I built on this, I'd log and diff that file like any other agent edit. Now, the rule your agent dropped. If it went at a
compaction, the cut happened in the harness at a length the tool sets. A CLM moves the cut into the model. On the paper's numbers that holds accuracy and saves compute at the price of a new
cache trick and a new safety risk under the hood of your current tools, nothing changes yet. My rule for now, if a long session depends on something you said early, say it again after a compaction.
Cloud codes docs already tell you it may be gone. My call. If you build agent harnesses or run inference servers, read this paper and that repo because the compute numbers are the part worth
taking. If you just use clawed code or codecs, it's one to watch and there's no setting for it yet. Who would you rather have holding the scissors in your next long session? The harness or the model?
And if a compaction's ever cost you a rule, tell me which rule it was.