Run "Kimi K3" on a Laptop With 32 GB Ram (No GPU Needed)

2.78 trillion parameters, the largest set of open weights anyone has published running on one laptop, not a distilled copy, not a pruned copy. The actual checkpoint Moonshot released doing actual work on a MacBook Pro. It needs 29 GB of RAM to start, less than a lot of gaming rigs have sitting idle right now, and it is slow. Half a token per second, about 30 seconds to speak the sentence you just heard me say. So, why did this repo pull 400 stars in 4 days and land on the front page of Hacker News? Three numbers explain all of it. 29, 17, and 0.5. Hold on to those. 29 is the RAM floor, the number the title sells. 17 is the number the title leaves out, 17 GB. That is how much this engine reads off your SSD to produce one single word per token, not per prompt, not per session. 92 layers, 16 experts each, all of it fetched from disk. The engine is called waste, 6,000 lines of C. No Python, no CUDA, no BLAS, one binary and one file on your drive. One developer wrote it in 4 days, starting 2 days after the weights went public. Let me show you how it works. Start with the model, because the model is why this is hard. Kimmy K3 is a mixture of experts, 2.8 trillion parameters in total, but only about 104 billion of them fire for any given token. 93 layers, 896 experts per layer, of which the router picks 16. Moonshot announced it on the 16th of July and put the weights up on the 27th. A million tokens of context and a bespoke license with a revenue clause in it, so call it open weights rather than open source. The download is 1 and 1/2 terabytes across 118 files. And it is not a curiosity. On Browsing Comp, it leads the field at 91.2. Terminal Bench 2.1, 88.3, within half a point of the best closed models. It loses the coding crown to Fable 5 and GPT 5.6, but it is in the fight, which is what makes the hardware bill hurt. To serve K3 the way Moonshot serves it, you want 8 B300S, about 2.3 terabytes of video memory. Their own deployment notes suggest 64 accelerators. The floor people quote is 8 H100S. Rent 16 H200S at the going rate, and you are near $66 an hour, call it 48,000 a month, or skip the hardware and pay their API at $15 per million output tokens. The weights landed on the 27th of July. This repository was created on the 28th at 9:00 in the evening UTC. One day later, 4 days after that, it was on the front page, and the code was still being rewritten underneath it. Here's the card, exactly as GitHub served it to me, Apache 2.0, written in C. Three contributors. That description is the entire pitch, and the whole video is a test of whether it holds up. Marco Bambini wrote 127 of the 130 commits. He founded SQLite Cloud and SQLite.ai, so this is a database person walking into inference, which turns out to matter, because the whole design is a storage design. Now the trick, and it starts with something true of every mixture of experts. The model activates about 4% of itself per token. 16 experts out of 896 is under 2% of the experts in any layer. The other 880 sit there doing nothing. The readme says it better than I can. Almost all of that weight is idle at any instant, and idle weight does not need to be in memory. That sentence is the entire product. Everything after it is engineering, so the engine cuts the model in two. 27.28 GB stays resident in RAM permanently. The embeddings, the attention layers, the routers, the output head, quantized to 4 and 8 bits. Everything else lives on the drive and gets fetched the moment the router asks for it. And that resident trunk is almost the entire RAM floor. 29.05 GB at 4,000 tokens of context, and 27 of those are the trunk. The engine refuses to start below it, which is an unusually disciplined thing for a piece of software to do. One caveat the headline leaves out. That floor is measured at 4,000 tokens. Push to 128,000 and it becomes 35.6. Use the full million tokens of context K3 advertises and you need 83 GB of RAM. The 29 GB number comes with a short conversation attached. Here is one token end to end. The router runs, picks its 16 experts, the engine reads them off the drive, multiplies, and moves to the next layer. Then it does that 92 more times before a single character appears on your screen. 16 experts * 92 layers is 1,472 reads. Every one of them a real trip to the SSD. Add the bytes up and you get 17 GB. That is the price of one word. This is where the database instincts show. Each expert record stores its gate up and down matrices adjacent to each other aligned to 4 KB. So routing to an expert costs exactly one read, not three, not a seek per matrix. That layout decision is worth more than any amount of clever threading. The experts are compressed hard to make the read smaller. Residual vector quantization. Three stages of 256 entry codebooks over eight dimensional vectors landing at three bits per weight. Moonshot trained the model with quantization aware training on the experts, which is why this survives at all. And the full matrix is not rebuilt in memory. The engine multiplies straight out of the codebooks. Build a small table of partial dot products once, and every expert row becomes three table lookups and two additions. There is no decompression step to pay for. Then it does the thing that started the argument on Hacker News. It refuses the operating system page cache. F, no cache on macOS, O_DIRECT on Linux, no buffering on Windows. Every read goes around the kernel, deliberately. The reasoning is worth quoting. With a container smaller than RAM, the kernel would cache everything, and hit rates measured that way are a fiction that does not survive contact with a 982 GB model. Benchmark on a small model, ship on a huge one, and your numbers described a machine nobody is running. Put shorter, a cache you do not control is not a cache, an engine should stop asking for memory before the OS starts taking it back. If you have ever watched a machine start swapping at 3:00 in the morning, you already know why that line is in there. Attention gets the same discipline. By folding one projection into the query and output paths, the engine caches a 512-wide latent instead of full keys and values. 11 and a quarter GB of cache becomes 0.21, 53 times smaller, and without it, the RAM floor would not fit on a laptop top at all. Then, on the 31st of July, one commit. Read the next expert while computing this one. Two threads reading ahead, so the disk and the arithmetic overlap instead of taking turns. That single change took K3 from about 0.32 tokens of second to 0.5, 60% faster in a day from overlapping IO with compute. If you read the blog post from the 30th and the readme from the 1st, they disagree, and this commit is why. Which tells you exactly what kind of workload this is. 82 and a half percent of decode time goes to the expert path. 53 and a half percent is disk reading alone. The actual matrix multiplication is 20%, and all of attention is under three. So, this is a storage problem wearing a machine learning costume, which means the usual instincts are wrong, and I got the next one wrong, too. Give it more RAM, cache more experts, read the disk less, go faster. That is obviously how it should work. The benchmark table in the repo says otherwise, and it is the most interesting table in the whole project. 32 GB of budget gives you half a token per second. 46 gives you 0.54, the best result on the board. 52 GB gives you 0.04. 58 gives you 0.02. Read that curve again. Going from 46 to 58 GB of cash budget improves the hit rate and makes the engine roughly 25 times slower. More memory, dramatically worse. Because past the point the operating system takes the memory back. macOS starts paging out the trunk, the 27 GB that must never leave RAM leaves RAM. And now every token faults its own attention, waits back off the same drive it is already saturating. So the engine walks its own budget down 1 17 GB working set at a time until the whole thing fits under 7/8 of physical memory. That is steering around a cliff it knows is there rather than tuning for the best case. And underneath one token's working set, caching stops meaning anything. An expert cache for this token is evicted before the next token ask for it. As the readme puts it, the hit rate is not low, it is zero. Now the arithmetic that decides the ceiling, and this one is mine, not theirs. 17 GB per token means one token per second requires 17 GB per second of sustained sequential read. That is the floor for one token a second, not a target, a floor. The fastest consumer NVMe drives you can buy in 2026 do about 15 GB a second. So even with zero compute time and a perfectly empty cache, the best drive on the market caps out under 9/10 of a token per second. The hardware ceiling is right where the software already is. Want 10 tokens a second, which is roughly readable? 170 GB a second. That is 11 or 12 top-end Gen 5 drives strike together before you have bought a single stick of RAM. And at that point you have built a strange expensive computer to avoid buying a GPU. And it has to be internal. Put the same container on an external USB enclosure at 0.94 GB a second and one token takes 18 seconds. That is the difference between half a token a second and three tokens a minute and it is why the title says NVMe rather than SSD. On the laptop it was measured on, the internal drive does 12.78 GB a second and the engine is pulling about 9.9 of that, 77% of the hardware already. There is very little performance left to find in the code. Fine. But is this the real model or a lossy impression wearing its name? Every layer was validated against a PyTorch reference implementation. The final logits agreed to 3.6 * 10 to the minus six. The vision tower matches its Oracle to 2.3. Those are floating-point rounding differences. Whatever comes out of this engine is what would have come out of the original. An independent reviewer read the code on Hugging Face and wrote that the math is correct, the conversion is honest and the test suite validates against PyTorch Oracles. Zero network calls anywhere in the sea. For a four-day old project handling a one and a half terabyte download, that is reassuring. The same reviewer found the other side of it. Those three-bit experts carry about 19.4% error against the original weights against 4% from Moon Shots own format. The weights really are lossy. The answers measured at the logits survive it anyway. Then there is the thread itself. 235 points, 96 comments and two objections worth taking seriously. The first was blunt. One commenter called the headline claim outright nonsensical and did the arithmetic. 115 GB of dense parameters at native precision plus 25 GB of active experts, no way that fits. He was pricing an unquantized model. The trunk is stored at four and eight bits, which is how 115 GB becomes 27. And the active experts were not going to be resident. That is the entire point. The objection was arithmetic against a design it had not read yet. The second objection was about your hardware, and it is the one you will hear every single time this comes up. Streaming that much data, the argument goes, will destroy the drive's right endurance. It will not, and the reason is worth knowing. Endurance ratings, TBW and DWPD, are defined on rights, where it comes from program erase cycles damaging the cell. Reads do not consume them. Kingston, Kioxia, and the CMU paper all agree. And another commenter on the thread got there first. The model weights are read workload, not a write one. The electricity is a fair complaint, though. A 2025 paper argues that SSD offload from mixture of experts burns roughly 12 times more energy per token than keeping the weights in high bandwidth memory. You are trading somebody else's capital cost for your own power bill. A commenter costed the laptop at about $5 per million tokens in electricity, assuming 42 W sustained. The API charges $15 per million output tokens. So, it is cheaper if you value 550 hours of waiting at zero, because half a token a second is 22 words a minute. You read at around 238 words a minute. A 500-word answer takes 22 minutes to arrive, and you will watch most of it happen one word at a time. The same model on Moonshots own API runs around 35 tokens a second, 70 times faster. That gap is the whole trade, stated plainly. You are not buying speed, you are buying the fact that it runs at all on hardware you already owned, with nothing leaving the machine. One human detail before the verdict. Half the thread piled on the readme for reading like a language model wrote it. The author answered that he has written a programming language by hand, and now uses models to write better code faster. Both things being true at once is very 2026. And now the part the headline skipped, which I think is the most useful part. This is not the first project to do it, and it is not the only one. Deltafin has more stars than Waste does and runs full K3 on a single device, too, with about 114 GB resident and 25.8 GB read per pass. It scored 0.266 tokens a second on an M1 Max. So, Waste reads 34% less per token and runs roughly twice as fast on newer silicon. Faster and leaner, not first. The readme hedges the claim carefully and the thread found the counter example within hours anyway. The wider field is busier than one thread suggests. OLLM hits the exact same half a token per second streaming an 80 billion parameter model to an 8 GB graphics card. Same number, a model 35 times smaller, completely different bottleneck. K Transformers gets 13.7 tokens a second on a 671 billion parameter model, which sounds like it wins until you read the hardware. 382 GB of DRAM and two server CPUs. llama.cpp manages four and a half on that same box. It is not a new idea, either. Back in 2023, FlexGen ran a 175 billion parameter model on one 16 GB T4 at about a token per second and people called it a party trick then, too. Three years later, this runs a model 16 times larger at half that speed. The models grew 16-fold and the technique held. That trend line is the actual story here and it is pointing somewhere and it is not K3 specific. The author points the same engine at a 48 billion parameter model and gets 10.7 tokens a second out of a 19 GB container using 1.87 GB of RAM. That one is not a demo. You could use that today. So, what is it actually for? Not chat. Something you send a hard question to and read in the morning. An agent that has all night. The author's own line is the fairest summary. The interesting part is not the speed, it is that the whole thing is in reachable range. One last thing, because it says something about all of this. He posted this repo to Hacker News twice. The first time, two days earlier with a title describing the idea. It got seven points and three comments. Same code, same author, same week. The second post led with the numbers instead of the idea and got 235. A commenter on the dead one wrote, "What a gem that did not appear on the front page." The model that needs 64 accelerators now also runs on a laptop. Badly. Both of those are true at the same time and only one of them was true a month ago. If your machine has 32 GB free and a fast internal drive, the barrier is a 1 and 1/2 TB download and a few hours of patience. That is waste. Links to the repo, the thread, and the model card are in the description. Tell me what you would actually run overnight at half a token a second.