This 6k AMD PC DESTROYS NVIDIA's DGX Spark?!

0:00 A $5,999 box with a 32 GB graphics card. Runs Deepseek V4 Flash about 3.4 times faster than Nvidia's DGX Spark. That's in the maker's own test, but the model has 284

0:14 billion weights and the file is about 100 GB. A 32 GB card can't hold it. You can copy part of the idea on your own PC. But first, how does a small card help with a model three times its size?

0:26 The box comes from a startup called Loose Box. Its main memory runs at 273 GB per second, the same as the Sparks. Loosebox says the speed comes from where each part of the model sits in memory.

0:37 Only a small share of the model works on any one token. So, the small card only has to hold the parts that work most often. The usual fix is one big pool of shared memory. It holds the whole file,

0:48 but its red speed limits how fast the model runs. The Spark works this way. Loose Box's compressed build is 198.29 GB file. Most of it is experts, which are blocks of weights that only some

1:00 tokens use. The build stores them in two or three bits each, and it stores the weights every token uses in four bits. That's small enough for a 128 GB Ryzen system to load the whole file without a

1:11 split. The next figure is an estimate that I calculated from the sourc's numbers. The 98.3 GB file holds 284 billion weights. So, the average weight is about 2.8 bits. So 13 billion active

1:24 weights read about 4.5 GB for each token. A memory at 273 GB per second, then has a ceiling near 60 tokens per second. Caching and the real mix of weight sizes also change that number.

1:36 The Ryzen AI Max Plus 495 chip in the loose box 0495 has a big pool of 128 GB at 273 GB per second. The R9700, a Radeon AI Pro card, has a fast pool of 32 GB at 640 GB per second. The first

1:55 200 units cost $5,999 for the 128 GB model. A $192 GB version cost $2,000 more. Delivery is expected in January 2027. Loosebox's blog describes an earlier prototype. That

2:11 prototype used a Ryzen AI Max Plus 395 with 256 GB per second of memory. So, the Blogs chip may differ from the 495. The Spark has one pool at 273 GB per second. The Loose Box adds the R9,700's

2:27 fast pool at 640, but that pool's 32 GB is about a third of the file, and it also holds a draft model and the working cache. So, the R9,700 only helps if the right weights sit in

2:39 it. A link joins the two pools. No source gives it speed. So this video assumes it's slow. Loose box's allocator counts the link rate as a cost. So the design keeps the weights in place and

2:50 sends only small results across. In a mixture of experts model, each layer holds a router in many experts, which are blocks of weights. Loose box's guide says V4 flash has 256 routed experts and

3:03 one shared expert across 43 layers. For each token, the router picks six of the 256 experts. Deepseek's card says 13 billion of the 284 billion weights work on each token. That's about 4.6%.

3:16 Some parts run on every token, including attention and the shared expert. Loose box calls this group the dense path. The routed experts are most of the file, but each one runs only when the router picks

3:27 it. So, the two groups need different places in memory. Some routed experts get picked more than others. A research paper on a different model, Mixtrol, saw one expert in the last layer reused 15

3:38 or more times in one sequence. In models with one 128 experts per layer, fewer than 5% of experts activate for small batches. This evidence comes from other models, but hotness depends on the

3:50 workload. The same paper found that after more than 1,000 sequences with Mixtrol, reuse counts even out. So, loose box profiles real routing first. You capture routing on representative

4:01 requests into a hotness file and then you serve with that file. In the placement, the R9700 holds the dense path, the most selected experts, an 11.3 GB draft model, and the working cache.

4:13 The Ryzen chips big pool holds the rarelypicked experts. Each expert sits on one chip only. Follow one token through one layer. Attention runs on the R9700.

4:24 The router picks six experts. The picked experts on the R9700 run there together with the shared expert. At the same time, the picked experts in the big pool run on the Ryzen chip. The same steps

4:37 repeat for each of the 43 layers. Two things cross the link at each of the 43 layers. The Ryzen result copies straight to the R9,700, which combines the two results. The DS4

4:48 guide also says a state tensor of 4* 4,96 numbers per token crosses the link at each layer boundary. If each number is a 32-bit float, that's about 64 kilobytes. But the width is my

5:00 assumption. The weights never cross, and that's far less data than the gigabytes of weights each token reads. The R9,700 also keeps the cache of earlier tokens and picks the next token. How many

5:12 experts go on the R9,700? Loose box's allocator adds the next hot expert only if that lowers the longer of the two branch times. The link rate counts as a cost in that comparison. So,

5:24 the allocator balances the time of both branches. An even split would leave the R9700 waiting. According to Loose Box's blog, the R9700 reads memory about 2.3 times faster than the Ryzen chip. 640

5:38 divided by 273. So, with equal work, it finishes first. The R9700 holds at most about a third of the file. But its experts get picked most often, so it does more than a third of the reading.

5:50 So, the split is lopsided by design. The blog says only the most selected experts go to the R9,700. No source says how many that is per layer or what share of picss they catch.

6:01 A dense model gets less from this trick. It uses every weight on every token. So it has no rarely used experts to leave in the big pool. Tech Times reports this reasoning. No benchmark here tests a

6:12 dense model on this box. The method has limits. Experts in adaptive formats can't be split. Only fixed codebook rock MFP files support this split. The guide also says page concurrency refuses to

6:24 run with draft models and some other features. The best placement also depends on the routing of your own workload. So, how much faster is it? Loosebox's blog ran a serving test with

6:34 a 2K token prompt and 128 output tokens. The test had two warm-up runs, three measured runs, and temperature zero. The median was 51.1 tokens per second against 14.09 for one DGX Spark. That's

6:48 3.63 63 times in a context matched sweep at 2K, 4K, 8K, and 16K with prompt caches off. The speedups ran from 3.09 to 3.60 times. Tech Times reports an average of 47.75

7:05 against 14.09, which is 3.39 times. The README's headline of 86 tokens per second is a third figure at 2K context. These are different tests, so don't read them as one measurement. The readme

7:18 lists the Ryzen chip alone at 42 tokens per second at 8K context, but the hugging face page for the 98.29 GB build lists only 18.1 to 22.3 tokens per second on that chip alone. The two

7:32 single chip figures differ, and the sources don't say why. So, the size of the single chip baseline and of the gain from the R9,700 is unclear. The 86 at 2K isn't a clean

7:43 pair with the 42 at 8K because the contexts differ. Placement isn't the only change in the 86 result because Loose Box also used speculative decoding. A small draft model 11.3 GB

7:56 here guesses several Nex tokens and the main model checks the guesses together. Loose box reports a draft acceptance of 0.97. So almost all guesses were kept. The 86 is above the 60 estimate and two

8:09 things could explain that. The R9700 reads its share of the weights at 640 gigabytes per second, and one read of the weights can check several guest tokens. No source says how much each one

8:21 adds, so placement may explain only part of the 3.4 times. The two sides also ran different model files. The Spark ran a Q2 compressed build. Loose box ran its own rock MFPX build. The blog says each

8:34 side used the setup tune for its own hardware. So, the ratio compares formats and software as well as hardware. A community repo dated the 21st of August 2026 measured 44 to 47 tokens per second

8:47 on one Spark. It used a 3-bit checkpoint that keeps 216 of 256 experts per layer with the other 40 removed at up to 384K context. That isn't the same setup as Loose Boxes, so it isn't a fair match,

9:02 but it suggests 14.09 09 reflects the Q2 build and its software. It isn't a ceiling for the Spark. Loosebox's blog tested a 128 GB Spark at $4,699, the earlier price. Crypto briefing

9:17 reports that model now cost $6,950, which is more than the $5,999 Loose Box. It also reports a new 64 GB Spark at $4,999 on sale the 23rd of October. That model

9:32 can't hold a 98 GB file, so it wasn't the one measured. Every throughput figure comes from Loosebox's own testing team. No independent reviewer has tested production hardware, and the first unit

9:42 to ship in January 2027. Nobody has run the same quantization on both machines. Until an independent review test it, treat the 3.4 times as the makers claim. The headline is also a decode figure,

9:55 which means how fast new tokens appear. Prompt processing is a separate number and the readme lists 788 tokens per second at 2K. One outlet reports the Spark reads prompts faster at about

10:07 1,077 tokens per second. That figure is secondhand and unchecked, so the claim covers how fast the answer streams. It doesn't cover how fast the box reads a long prompt. Loose Box's readme also

10:18 documents a three- tier mode that adds an SSD to the R9,700 in the Ryzen chip. It lists 25 to 26 tokens per second on code for V4.1 flash, not V4. You don't need two chips

10:31 to copy the idea. A community guide for Llama CPP splits a mixture of experts model into always active parts and routed experts. Attention, the dense layers and the shared expert are always

10:42 active, so they stay on the GPU. The routed experts are most of the size, so the guide puts them in system RAM. Then it moves as many experts back to the GPU as your VRAM allows. The idea matches

10:54 loose boxes. Keep the weights that every token uses in the fastest memory. Run llama server with your model. Add the NGL flag with 999 to put all layers on the GPU. Then add the OT flag with exps

11:07 equals CPU to keep the expert weights in system RAM. Or use the NCPU MO flag with a number. That number sets how many layers keep their experts in system RAM. Then change the number and watch your

11:18 VRAMm and tokens per second. This version is simpler and it differs in two ways. The CPU computes the experts in RAM where loose box uses a second GPU. The guide picks experts by layer where

11:30 loose box uses profiled hotness. So you get the placement idea without the second chip or the profiling. No source here gives a speed gain for this home setup. Before and after the change,

11:40 measure your own tokens per second. The decision depends on your model. If you run a mixture of experts model, keep attention and the shared expert on the GPU. Then fill the rest of your VRAMm

11:50 with experts. If your model is dense, expect less from this trick. If you plan to buy a loose box, I'd wait for an independent review. Both boxes have the same 273 GB per second of main memory.

12:02 Loose box adds a faster card and a rule for what goes on it. That rule, speculative decoding, and a different file all feed the 3.4 times. Here's the open question. If both machines ran the

12:13 same file and software, how much of the 3.4 Four times would remain.