Xiaomi Shrunk 12GB of AI Memory to 2.7GB (MiMo-V3)

0:00 Your agent runs npm test eight characters. Back comes 20,000 tokens of logs and stack traces. It typed eight characters and got back a novella. And before it can type the next command, the

0:11 model has to read every one of those tokens. Then it runs the next test and the whole thing happens again. Shomi's fully luo describes exactly this loop in her announcement. Each round a short

0:22 action returns a long observation that has to be prefilled and the context keeps growing. So three things squeeze you at once. the compute to read the new chunk, the memory to remember

0:32 everything, and whether the model can still find the one line that matters. Her team's answer is high sparse, too. And she says it's the core of Mimo V3's new architecture. At a million tokens,

0:43 it shrinks the model's memory of the context from 12.09 GB to 2.69 and cuts prefill compute about five times. And retrieval, which you'd think would get worse, actually improved on their 256K

0:56 test. That's measured against Xiaomi's own previous design, and it's still a big jump. It also has a strange twin. 12 days before this paper landed on archive, Deepseek shipped V4, one flash

1:09 with the same two tricks inside. By the end, you'll know why that happened and why copied is the wrong word for it. First though, you need to see what's actually being skipped. Picture the

1:19 model as a tall building. Each floor is one layer, and Xiaomi's test model has 49 of them. Every token, every word fragment of those logs walks in the front door and gets read on every floor

1:29 on its way up. Each floor keeps a filing cabinet and files a card for every token it reads. The card's label is the key roughly what this token's about. Its contents are the value what the token

1:40 contributes when it's relevant. When the model writes its next token, each floor checks its question against the labels and pulls what matches. That cabinets the KV cache. It saves the model from

1:51 rereading the whole conversation every turn, but it grows by one card per token on every floor that keeps one. In Shiaami's previous design, a million tokens came to 12.09 GB. Prefill is the

2:02 filing itself. When those 20,000 tokens of logs arrive, every floor has to read them and file its cards before the model can say anything. In an agent loop, you pay that bill every round. That previous

2:14 design is hybrid S swa, short for sliding window attention from the MIMO B2 series. Nine of its floors do full attention and keep a card for everything. The rest mostly watch a

2:25 sliding window of recent tokens. It's the yard stick for Xiaomi's numbers along with the original high sparse. Trick one is KV bridging. Split the building partway up roughly in the

2:35 middle. The lower floors read every token the normal way. The top floor of that lower section becomes a handoff floor and it leaves behind a set of notes. Its understanding of every token

2:46 so far up top. The full attention floors still keep cabinets, but they don't read the tokens themselves. The handoff notes go up a chute and each of those floors fills its cabinet straight from them. So

2:58 once the lower section finishes, prefills done. It gets to go home at the handoff floor. The paper calls the lower section the self decoder and the upper one the cross decoder. and it puts the

3:08 idea plainly. Prefill can exit after the self-decoder. That layout comes from a 2024 paper called Yoko, short for you only cash once. You do give something up. The upper floors never form their

3:20 own view of each token during prefill. Their picture of the past is whatever the handoff floor wrote down. Xiaomi's betting that's good enough, and the retrieval test will show how that bet

3:30 went. Trick two, KV reuse works inside each block of floors. A block starts with one full attention floor that reads everything and keeps a proper cabinet. Above it sit a few sparse floors which

3:42 only look at a small selection of tokens. While the full floor reads, it works out which tokens matter most and writes them on a short list. The sparse floors above don't build cabinets at

3:52 all. They walk downstairs, borrow the full floors cabinet, and read only the cards on that list. The sparse floors are the flatmate who never buys groceries but only eats what's on the

4:01 list. The trick works because the full floor already did the hard part. Deciding what's important. Borrowing the list saves the choosing and borrowing the cabinet saves the memory. Ruse

4:11 itself isn't new. Shiaomi published it in February as the original high sparse. Sparse floors borrow the list and the cabinet from the full floor below. Highspars 2 keeps that and makes two

4:22 smaller changes. Change one is how cards get picked. Picture the cabinet holding binders. The earlier approach pulled out whole binders. Highsparse 2 picks individual cards which the paper calls

4:33 token level selection. If the line you need is buried in a binder of noise, you get the line without hauling the noise. Change two is recent context. A sparse floor always needs the last few things

4:44 that happened. And that used to come from a separate sliding window branch with its own storage. Now it's a fixed shelf in the same borrowed cabinet. The most recent one, 28 tokens, always kept

4:54 next to 1,024 picked ones. So, a sparse floor facing a million token context reads 1,152 cards, about a tenth of a percent of the cabinet. It gets away with that because

5:05 the floor below already read everything and told it where to look. Now, count cabinets. Only five of the 49 floors do full attention, down from nine in hybrid SWA. Sparse floors borrow instead of

5:17 building, and the whole upper section skips reading during prefill. Fewer cabinets, less filing. Add a million tokens that buys you this. The KV cache is 2.69 GB. The original HighSpar needed

5:30 6.72. Hybrid S SWA needed 12.09. That's roughly 4 and a half times smaller than Xiaomi's previous design. Prefill compute drops 5.02 times against hybrid S SWA and 2.92 times against the

5:45 original highsparse which already had reuse. These are computed flops. a count of arithmetic operations on an 80 billion parameter research model with 3 billion active. They aren't stopwatch

5:56 timings and they aren't a comparison with anyone else's model. Does all that borrowing make the model worse at finding things? You'd expect it to. On Ruler Dagi 2, a long context retrieval

6:07 test at 256k tokens and after a light post training stage, Highspars 2 scored 58.45. The original Highspars scored 32.61 61 and hybrid S swa 35.74. So on this test, the borrowing didn't

6:22 cost it the one line that matters. One catch. The cash and compute numbers are at a million tokens. The retrieval table stops at 256K. A quarter of that. It's the long context version of the test

6:34 passing on a smaller fixture. Hold on to that gap. Now leave Xiaomi's building and rewind to the 10th of September. Deepseek released V4 one flash a 552 billion parameter mixture of experts

6:46 model. So only a slice of it runs for any given token. It has a million token context and it activates 8 billion parameters while processing the prompt and 16 billion while generating half the

6:57 active parameters for reading the prompt. It's what you'd expect if prefill stops halfway up the building. Deep Seek calls this the causal encoder decoder. Its upper layers don't build

7:06 their keys and values themselves. They're projected straight from the middle layer's notes, what the paper calls its hidden state. Move for move, it's KV bridging, a handoff floor in the

7:16 middle and upper cabinets filled from the notes it sends up the shoot. The second trick is in CSA 2, DeepS sparse attention. Its reuse mode lets layers borrow the main cabinet and the latest

7:27 top K picks, the short list of most relevant tokens from the layers below. Same staircase, same borrowed list. Shami calls the first trick KV bridging. Deepseek calls it a causal encoder

7:38 decoder and Yoko called it a decoder decoder. That's three names for one shoot. Turns out the caching was the easy part and the dates are close. Deepseek released on the 10th of

7:48 September and Xiaomi's paper went up on archive on the 22nd of September. Did someone copy? No. And the citations show why. Deepseek's report says its encoder decoder is inspired by Yoko, a Microsoft

8:01 and Chinua paper from May 2024. Yoko's pitch was this exact move. A self-decoder builds the cache once, a cross decoder reuses it, and prefill ends early. The reuse half has an even

8:12 clearer trail. Deepseek's report credits high sparse by name for letting sparse layers reuse the KV cache of dense layers. It's Xiaomi's own February paper published 7 months before either

8:24 September release. The fair version then same idea, shared ancestors. Both labs built on Yoko's bridge. For the reuse half, Deep Seek credited Xiaomi's February paper in writing. Then each put

8:36 out its own version of the pair 12 days apart. And the trail runs both ways. Xiaomi's new paper cites Deep Seek's report too for a different trick. Could you say whose version is better? Not

8:47 from these papers. Deepseek reports 890 bytes of cash per token, a figure that already counts squeezing numbers down to four bits called FP4. Xiaomi reports total gigabytes at a million tokens on a

8:59 different model. Two rulers, two buildings, no fair side by side. Did Xiaomi kill it against its own last design? Yes. About four and a half times less cash, five times less prefill, and

9:12 better retrieval at 256K. If you run or fine-tune models for agents that pour long tool output back in, this is the family to watch. When two labs land on the same pair of tricks

9:23 and cite each other's work along the way, that's a decent signal. If you just call an API, there's nothing to change yet. MIMO v3 weights haven't shipped, and nothing confirms the production

9:34 model matches this ADB test setup. When the weights do land, the question I'll be checking is simple. Does retrieval hold up at a full million tokens?