Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Nanbeige's 33B model claims a 23-point win over Qwen 27B on Deep Suite, but its KV cache is about three times larger per token, cutting usable context to roughly a third on a 24GB GPU. The benchmark table shows mixed results, with Qwen leading on Swebench Pro. For agent workflows on a single 24GB card, Qwen remains the safer default until independent verification of Nanbeige's claims.
Key points
Nanbeige 33B claims a 23-point win over Qwen 27B on Deep Suite (65.5 vs 42.5).
On Swebench Pro, Qwen 27B edges Nanbeige 33B (61.7 vs 60.3).
Nanbeige 33B uses about 208 KB KV cache per token, while Qwen 27B uses about 64 KB per token.
Nanbeige 33B has 52 layers with full KV cache, while Qwen 27B has 64 layers but only 16 use full KV cache.
On a 24GB GPU, Nanbeige 33B allows roughly 23,000 tokens of context, while Qwen 27B allows about three times that.
Tools mentioned
Techniques
- KV cache
- 4-bit quantization
- agent harness
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
Nan Bay's new 33 billion model claims a 23point win over Quinn's 27 billion on one coding test. On Deep Suite, it's 65.5 to 42.5. On Swebench Pro, Quinn edges it 61.7 to 60.3. Both rows come
from NanBay GU's own table. Every model at its highest thinking effort. The agent harness, the tool that steps the model through a task, changes test to test. So, these are results on paper and
the table doesn't say why the gaps point opposite ways. But there's a closer question for you. If Quinn's your coder on one 24 gig card, should you switch? Nanbbegu's own table gives it 18 of 20
rows. On your card, the same memory holds about a third as much conversation. The obvious culprit is size. 33 billion parameters against 27. The 4-bit files bigger, too. 20.8 GB
against 17.4. But once loaded, it stays that size. The conversation is what grows. Each layer keeps notes on every token, so it never recomputes the earlier ones. Picture a notebook per
layer, one line per token. Nanbay Guz config shows 52 layers. So every token adds 52 lines. Each line holds two lists of numbers, a key and a value for eight heads, 128 numbers each, two bytes a
number. Multiply it out, about 208 kilobytes per token. Quinn's model card shows 64 layers and only 16 keep a notebook that grows. The other 48 keep a fixed size summary and that's where the
notebook picture breaks. Run those 16 through the same sum about 64 kilobytes a token. That summary takes memory too and Quinn's model card doesn't say how much. A 24 gig card holds about 25.8
billion bytes. So the 20.8 GB file leaves about 5. That works out to roughly 23,000 tokens before the runtime takes its cut. And Pasquallay pill itary's write up lands on 20 to 25,000.
That goes fast once an agent paste in a stack trace. Divide one cost by the other about three times the notebook for those 23 points. For agent work on one 24 gig card, I'd keep Quinn as the
default until someone besides Nan Bega reruns that table. Would you give up 2/3 of your context for 23 points?