Qwen3.8-Flash-Next: Why 51B Parameters Never Touch Your GPU

0:00 On Monday, a folder went up on hugging face with no announcement attached to it. The blog post wouldn't appear for another two days. And inside it, among the weights, sits a small configuration

0:10 file. One line in that file gives away what the whole thing is for. The architecture and names isn't a 3.8 anything. It reads in plain text 4. That's the tell. This isn't a product

0:22 launch. It's the skeleton of a model that hasn't shipped yet handed over early on purpose. The numbers are odd, too. 125 billion parameters in the main model. 6 billion of them fire for any

0:33 given token. Fine. That's a sparse mixture of experts. And we've all seen that trick by now. Then there's a second number and it doesn't belong to a category you already know. 51 billion

0:44 more parameters. They aren't experts. They aren't attention. They aren't a neural network at all. They're a lookup table, a phrase book. And the people who built it would rather it wasn't on your

0:54 graphics card. That last part is the entire video, not the benchmark scores. the sentence about where those parameters are supposed to live because if it holds up the biggest single chunk

1:04 of a Frontier open model just stopped needing the expensive memory. So three things in order what the four changes actually are in plain language. Why every one of them got picked against the

1:15 training loss curve and what three unrelated people measured in the days after the weights appeared. The model is when 3.8 8 flash next from the trend team at Alibaba. We waits on Monday,

1:26 blog post on Wednesday, 28 pages of technical report alongside it. It's open weight and it is not open source. We'll read the license later because for a few of you that difference decides

1:37 everything. One bit of housekeeping because two things share that name and mixing them up will waste your afternoon. Flash. Next is the openweight preview you can download. The plain twin

1:48 3.8 flash is the hosted production version. A million tokens of context by default, built-in tools, 16 cents a million in that one you rent. But previews are cheap to announce and easy

1:59 to abandon. So the fair question is whether this team follows through. In September of 2025, they published a preview called Twin 3 Next. 160 days later, the hybrid design inside it was

2:11 carrying an entire production family and has carried every release since. So what's inside this one? Now for the architecture, four changes. attention, the residual stream, the embeddings, and

2:23 the optimizer in that order, and none of them needs maths. Attention is the part of a model that looks back at what you've already said. It's also the expensive part. The plain version

2:33 compares every new token against every earlier one, and that bill grows with the square of your context. So, three layers out of every four here don't do it at all. Those run something called

2:43 gated delta. Imagine one whiteboard. Every token walks past and edits. It rubs a bit out, writes a bit in. The board stays exactly the same size. It just keeps getting rewritten. That's

2:55 cheap and it's lossy on purpose. The board holds the gist of everything. What it can't do is hand back the exact sentence you typed 40,000 words ago because that sentence was written over

3:05 long ago. And that is why every fourth layer is different. That one gets real attention with the full history available. And its whole job is precise retrieval. The one thing a whiteboard

3:16 can't do. But real attention over a million tokens is the expensive thing we were avoiding. So it doesn't read everything either. It runs a cheap indexer first to decide what's worth

3:25 reading. The older versions of that trick score every individual token which works until the scoring becomes the expensive part. QSA chops the sequence into microblocks of four tokens and

3:36 scores the blocks instead. Fewer things to score, so the scorer stops being the bottleneck. It keeps the best 512 blocks, expands them back to 2408 token positions, and attends to those. Which

3:48 means at a million tokens of context, that layer is looking at roughly 2/10 of 1% of what you handed it, one position in every 488. And the compression only chooses where to look. The final maths

4:00 runs on the original uncompressed values. At a million tokens, they measure 7.6 times faster on prefill, 4.9 on decode, and label that properly. It's Twin's own benchmark on Twin's own

4:13 kernel and it measures the attention module, not your prompt end to end. Second change, and it's the one I'd have skipped if I hadn't read the ablation. The residual stream is the shared

4:23 notepad every layer reads from and writes to one notepad 48 layers deep. So everything written early gets buried under everything written late. So they widened it. One stream became four

4:34 parallel lanes, which by itself is a known idea. The new part is what sits on the read. Every layer now reads through a gate that decides per channel and based on what's in front of it how much

4:44 to take from each lane while writing back gets one small number per lane. Read finely, write coarsely. Then they looked at what the four lanes were doing. And one had turned into a highway

4:55 and across five separate checkpoints, exactly one branch ends up carrying the long connections. A typical jump of about 11 layers against three and a half for the other three. The first layer's

5:06 output arriving deep in the network went from 2% to 14. That lane wasn't designed. They handed the model four identical branches and it built itself an express route from the front of the

5:17 network to the back. The total amount of cross layer information barely moved. What changed is where it goes. Third change. This is the one you'll feel. And it isn't Chen's idea. They say so

5:27 themselves. Google's Gemma 3N did per layer embeddings first. Deepseek published engram in January and open sourced it. Chen is the third lab to land on the same answer, which is better

5:38 evidence than any of them being clever alone. A normal embedding table looks up one token and hands back a vector. That's the very first thing that happens to your text before any layer. An engram

5:49 table looks up the last two or three tokens together. So instead of asking what does bank mean, it asks what does the river bank mean? And it has an answer stored. That's 20 million stored

5:59 phrases in one layer near the front. The model stops reconstructing the boring repetitive local stuff from scratch every time. So the six billion parameters that do fire get to spend

6:09 themselves on the hard part. And now the sentence this whole video is built on straight off the model card word for word. Embeddings they write are more amendable to offloading than mixture of

6:19 experts. Indexing with short engrams makes parameter scaling highly efficient for memory constrained accelerators. That is an engineering decision, not a paper sentence. A Frontier Lab is

6:30 telling you they grew the model along the one axis that doesn't have to sit on the accelerator. Memory constrained accelerators. That's your machine. It's in the release notes. It works because

6:40 the address is knowable ahead of time. Your last two tokens are the key. So nothing has to be computed to find which rows you need. And the fetch can start early and finish while the first layer

6:50 is still busy, which is why the table sits at layer 2, not layer 20. That's three of the four. Now for the last one, and it's the least glamorous. It's also why the other three were affordable. So

7:01 stay with me for 90 seconds. They moved most of the training onto an optimizer called Muon. And this is the interesting bit, not all of it. The big two-dimensional weight matrices get

7:11 muon. The embeddings, the router that picks experts, and the small projections inside that residual gate stay on atom W because for those shapes, the new one doesn't help. The gate bought them

7:22 something else. And they proved it by trying to break the model. Hold the learning rate at four times optimal, the standard way of forcing instabilities out early. And the old configuration

7:31 spikes 183 times per 10,000 steps and lives on the gradient clipper. The new one records zero spikes, not fewer, zero. So then they deleted a step that is standard practice everywhere. Ramping

7:44 the batch size up gently at the start of training bought them nothing measurable and cost 18.8% more optimizer steps. 18% of the largest compute bill in the building paid for a habit. It all adds

7:56 up to about a third of the previous flagships active parameters, a third of the tokens, and roughly a ninth of the compute. So far, four architecture changes. A perfectly normal paper. What

8:07 makes this report worth your time is what they use to choose between them because in it the training loss and the benchmark scores keep disagreeing and every time they disagree, the loss curve

8:17 loses. Take their residual table. Making that gate data dependent improved the training loss by 2000s on any normal reading that's noise and you dropped the extra complexity. The same change was

8:28 worth two full points of average benchmark score. So which one do you believe the curve or the tests? They believe the tests shipped the gate and wrote in their own report that loss

8:38 alone would have undersold it. Then it happens in reverse. They tried making the residual read sparse only the top two lanes instead of four to move fewer bytes. The loss was unaffected, the

8:49 benchmarks unaffected, a free win. It fell apart after post-training. Their words, a case where pre-training metrics alone would have led to the wrong decision. They didn't ship it. Third and

9:00 fourth, growing that phrase book lowers the training loss every single time smoothly all the way up. While downstream accuracy just stops and wanders sideways, and dropping

9:10 positional encoding looked completely free in pre-training. After post-training, the model wouldn't stop generating. It kept going past the end forever. That's four times in one

9:20 document that the headline number was pointing the wrong way. The villain isn't a company and it isn't a benchmark. It's the habit of trusting whichever metric is easiest to plot.

9:30 These people built the expensive evaluation loop instead. Which brings us back to the lookup table because a sentence on a model card is worth nothing until somebody independent

9:39 measures it. Three groups did inside three days and they're worth taking one at a time. First, the serving engine. The SG Lang team moved the table into pinned host memory and gathered those 16

9:51 rows per token with a small kernel on its own stream, overlapping the first decoder block. Then they measured what it cost on a 4-way H200 node. Model weights per graphics card fell from

10:02 about 84 GB to 60, and room for cache context went up 78%, roughly twice the conversation you can hold before anything gets evicted. and throughput moved by 7 hundreds of 1%. Which is

10:15 nothing but the number that made me sit up is the next one. Four fixed prompts came back with identical output tokens and the recorded log probability trace matched exactly two. That's bit forbit

10:26 the same answer with 23 GB moved off each card on four prompts. So don't overread it, but for those four, that isn't a trade-off with a small cost attached. It's free. Second measurement.

10:38 And this one is my favorite because it didn't come from a lab. Somebody on hugging face going by the handle King Jones 777 published three quantized builds for stricks Halo AMD is unified

10:49 memory part. The one in those little mini machines that keep selling out. The three differ in one way. How hard the lookup table is squeezed. On disk, they're 88, 98, and 113 GB. But strip

11:02 the table out of each and what's left is the same size in all three within a fifth of a gigabyte. as they put it, one model plus a differently quantized lookup table which produces a

11:12 prediction. If the table really never reaches the accelerator, all three should use identical graphics memory despite a 25 GB spread on disk. And measured, they do 63 GB at load, all

11:24 three tiers at every context length they tried. Even at 131,000 tokens of context, it's 67. So on that machine, 46 GB of the file stays off the accelerator entirely. And they didn't take that on

11:37 faith either. They asked the colonel which pages of the file were actually resident. About 40 GB of it were sitting in the page cache. And the system's entire cache was 40.4. The page cache

11:49 was this file. Those pages are filebacked, which is the whole point. The operating system can throw them away and reread them when it needs the space, which is why they don't cost you real

11:59 memory. Third measurement isn't a benchmark at all. It's the timestamps pulled from the hugging face API this morning. 115 community repackages from 61 different accounts in the three days

12:11 after the weights went up. 31 of them Apple silicon builds, nine ports to Chinese domestic accelerators across seven vendors. And one does the obvious thing that hadn't shipped yet. It pulls

12:22 the phrase table out of the model file entirely and streams it off an SSD. It needs a custom loader. It won't run on stock llama.cpp CPP and it exists 2 days after release. The earliest conversion

12:33 is timestamped 20 hours after the weights appeared, a full day before the announcement blog went live. Somebody found the folder, read the config, and started quantizing before anyone told

12:43 them what it was. So, that's the good version, and I think it is good. Here's what a launch day thread isn't going to tell you. Starting with the benchmarks because they're the numbers that get

12:52 screenshotted. It scores 58.7 on Deep SWE, 62.5 on Swechen Pro, and 73.9 on Co-workbench. Real numbers, and Ken ran all of them on Ken's harness. To their credit, the footnotes say so. One of

13:06 those benchmarks is their own in-house test. Another reports the higher of two harnesses. A third was rescored after they corrected tasks they considered broken. None of that is dishonest, but

13:16 it isn't independent. and their comparison column is Claude Opus 4.6, which isn't the current flagship, but independent numbers do exist, whatever some write-ups claimed this week.

13:27 Artificial analysis has it scored, and 56 on their intelligence index is genuinely strong, level with Gemini 3.7 Flash and Gro 4.5 above Claude Sonnet 5. It is not the leading open model,

13:40 though, and you'll see people say that. Kim K3 sits four points ahead of it. Twin's own trillion scale flagship sits two ahead. This is a very good model that is not the best one, which is a

13:51 boring sentence and happens to be the true one. It's also chatty. Producing that score took 200 million tokens where the median model uses 110 and cheap per token starts doing a lot of work when

14:02 you spend nearly twice as many of them. Then the license, which I read rather than trusting the tag. It's called Twin Community 1.0 and it is generous. use it, modify it, sell it, host it,

14:15 fine-tune it. Two conditions. If you pass a hundred million monthly users or $20 million a month in revenue, you have to show the model's name in your product. The second one has teeth. If

14:26 your business is selling model access or an AI coding or office assistant, you need a separate license from Twin before you use it commercially. Internal use is exempt. Reselling isn't. Then the floor,

14:37 which is the one that decides whether any of this touches your life. It's 75 GB of memory for the smallest one-bit build, and at that size, it only reproduces the full model's top answer

14:47 about 80% of the time. So, the sensible pick is the 4-bit build at 112, which recovers 93. The offload paths aren't finished either. The VLM implementation needs 51 GB of host memory and currently

15:01 only runs on Nvidia. The SSD build needs a loader that isn't upstream. This is a preview and it's behaving like one. And one reply on the launch thread said the thing everyone was thinking. With RAM

15:12 prices this high, how can you call this local friendly? Which is fair. Nothing in this architecture makes memory cheaper. It makes slow memory usable. Those aren't the same sentence. So

15:23 here's where I come down. The scores aren't the story. And I don't think the Twin team believes they are either. They gave away the architecture, not the crown. The story is a lookup table, and

15:33 it wins for a specific person. Anyone whose machine is mostly slow memory, a big Mac, a Spark Box, an AMD unified memory chip, one graphics card next to a lot of system memory for you. This is

15:46 the first Frontier Scale architecture where the largest single piece was designed to live in the cheap memory. three receipts, 23 gigabytes moved off each card with bit identical output on

15:56 the prompts they checked, 46 GB of file that stays off the accelerator on a stranger's laptop chip, and the vendor's own release notes saying that was the point. Now, the concession, because it's

16:07 a real one, it loses the open weights leaderboard. The license would stop me shipping a product on it without asking first. The speed figures are theirs and they measure a kernel, not your prompt.

16:17 And 75 gigabytes is not most people's definition of local. I'd still take it because everything else near it on that leaderboard assumes you own the expensive memory, and this one doesn't.

16:27 That's the bet worth making. The next generation of open models won't compete on how many parameters they have, but on how few of them have to sit in the fast memory. If we get to next spring and

16:37 nobody has copied it, I was wrong. But there's a thought I keep circling. 28% of this model's parameters aren't a network at all. They're a phrase book that learns nothing while it's running,

16:48 sitting on the far side of a bus, being read 16 rows at a time. If the fastest growing part of a language model is the part that isn't a neural network, what is it we're actually building?