Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Turbo Field Fair runs a 26B parameter Mixture of Experts model on a Mac with only 2GB of RAM by keeping the shared core in memory and streaming the rest from SSD on demand. On an 8GB M1 Air it achieves 5-6 tokens/sec, but the prefill for a 1000-token prompt takes 37 seconds. The project is notable for publishing its own speed comparison against MLX (which is 2.5x faster but requires more memory) and for transparently documenting its limitations.
Key points
Turbo Field Fair is a custom Swift and Metal runtime that runs Gemma 4 (26B parameters, Mixture of Experts) on Macs with as little as 2GB RAM by keeping only the shared core (1.35GB) in memory and streaming the rest from SSD.
On an 8GB M1 Air, the model runs at 5-6 tokens/sec, which is slow but usable for reading-speed output on hardware that otherwise couldn't run the model at all.
The prefill phase (reading the prompt before generating) takes 37 seconds for a 1000-token prompt on the 8GB Air, a significant latency that limits use cases like pasting in a document.
The project's own benchmark table shows that on a 24GB M5 Pro, Turbo Field Fair achieves 31-35 tokens/sec, while MLX (Apple's framework) runs the same model at 76-82 tokens/sec on the same hardware—roughly 2.5x faster.
The key optimization was replacing memory mapping with explicit reads, which improved throughput from 0.5 to 4 tokens/sec by fetching entire experts in a single call rather than waiting for page faults.
About 41% of experts selected for one token are reused for the next token, so a cache of 16 experts per layer reduces repeated expert reads from 166ms to 88ms per token.
The project's memory footprint of ~2GB is real but incomplete: unused RAM on macOS is used as disk cache, so some expert reads actually come from RAM rather than SSD, and performance degrades under memory pressure (e.g., from 35 to 27 tokens/sec on an M5 Pro).
The project was motivated by rising memory prices (up 90% in a quarter) and the fact that Apple charges $400 for a soldered memory upgrade, making SSD streaming an arbitrage between expensive RAM and cheap storage.
Tools mentioned
Techniques
- Mixture of Experts (MoE) with 128 experts per layer, 8 active plus 1 shared expert
- Explicit SSD reads instead of memory mapping for streaming expert weights
- LRU and LFU caching of recently used experts (16 per layer)
- Prefill phase for prompt processing before generation
- Unified memory architecture on Apple Silicon (CPU and GPU share same memory pool)
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
Here's a MacBook Air from four years ago. 8 gigabytes of memory soldered down, no upgrade path. And here it is running a language model with 26 billion parameters, 14 GB of weights. Watch the
memory meter while it types. It doesn't move. 2 GB the whole way down the page. That's the claim, and it deserves some suspicion. 2 gigs of RAM on a laptop Apple stopped selling last year. All of
it comes from a project called Turbo Field Fair. one developer, a few weeks of work, 6,000 stars. Late July, it hit the front page of Hacker News and stuck there with 919 points. And the top
question in that thread is the one you're already asking. Where do the other 12 GB live? They're on the SSD and the engine reads them back off the drive for every single token that it writes,
which sounds like it should be unusably slow. On the 8 gig air, it runs at 5 to six tokens a second. slow, yes, but that's about the speed a person reads aloud on hardware with no business
running this at all. So, this is a tearown. How the trick works, what it costs, and the one number the project prints that most projects would bury. The trick only works because of how this
model is built. Gemma 4 is a mixture of experts. Instead of one big network where every parameter fires on every word, it's carved into small subnet networks and a router picks a handful of
them per token. Every layer here holds 128 of those experts, and the router picks eight of them. One more, the shared expert, runs every time, no matter what, which means the parameter
count in the model's name, is not the parameter count that actually does the work. At any given instant, about 85% of this model is doing nothing at all. That has been true of mixture of experts
models for years, and every engine still loads the whole thing into memory anyway because you cannot know which experts the router wants until it asks. Turbo Field Fair loads the part that always
runs and leaves the rest on the disc. That's attention, the routers, the shared expert, the embeddings. That core is 1.35 GB. It gets mapped into memory once and it stays put. Add the key value
cache, which is the model's running memory of the conversation so far, and you're at roughly 2 GB. The rest of the model sits in 30 files on the SSD, one for every layer, waiting to be asked
for. Then for every token at every layer, the router names its experts and the processor goes and fetches them. It goes straight into a buffer the graphics chip can already read because on Apple
silicon the processor and the graphics chip share the same pool of memory. And that's why this is a Mac project rather than a general one. And a custom Swift and Metal runtime written from scratch
rather than a wrapper around somebody else's. On a PC with a separate graphics card, you'd read from disk into system memory, then copy that across the bus into video memory. On a Mac, there's no
copy. The read lands where the GPU is already looking. So, that's the architecture. And on paper, it sounds easy. Now, the part where it stops being easy. Streaming weights off a disc isn't
a new idea, and a commenter in the thread said so. Memory mapping does it already. You tell the operating system a file is memory, and it pulls pages in when you touch them. The first version
of this engine worked exactly that way. It ran at half a token per second. So swapping memory mapping for explicit reads took the same simulation to just under four. Same disc, same weights,
eight times the throughput from changing nothing except who decides when to read. And the reason why is the most useful thing in the whole project. The operating system doesn't know what a
router is. It fetches a page when you touch it in small, careful chunks. And it can't start early because it has no idea what's coming next. An explicit read is a single call for a whole expert
and several can run at once while the graphics chip is busy elsewhere. A cold expert took 10 milliseconds through page faults and under three through a direct read. Same bytes off the same drive.
Different question asked. After that came reuse and this is the number that makes the whole design viable. The author measured it. About 41% of the experts picked for one token get picked
again for the very next one. Over half come back within two. So each layer keeps 16 of them in a small cache. That took repeated expert reads from 166 milliseconds a token down to 88. Then
switching the eviction rule from least recently used to least frequently used shaved another 8 milliseconds off. A small win measured and kept. So prediction was the obvious next step.
And it's the failure worth knowing about. If experts repeat across tokens, maybe this layer's picks predict the next layers and you fetch them early. They predict 7%. So that branch died
before it ever reached the runtime and the write up says so. So those are the four moves. Keep the shared core in memory. Read the rest deliberately. Cash what repeats and don't try to guess. 103
experiments are logged in that repo. Wins and losses. And the biggest win of all wasn't about the disc at all. Rewriting the expert maths for the graphics chip took that phase from 239
milliseconds down to 60. Which brings us to the number I actually wanted to show you. The project published a breakdown of a single token step on the 8 GB air. 162 milliseconds split four ways. And
that split explains the entire design. Expert reads off the drive 83 milliseconds. The command Q waiting on the graphics chip 55. The output layer that turns the last hidden state into a
word 14. Everything else the runtime does about 10. So half of every token is disk and a third of it is the machine standing still. The arithmetic, the thing you bought a graphics chip for, is
a rounding error inside its own budget. And the project labels that break down a diagnostic pass rather than a speedbound, which is a caveat most projects would not bother adding.
There's a worse number and the project prints that too. Before a model answers, it has to read your prompt in one pass. That pass is called prefill. And on the 8 GB air, a th00and token prompt takes
37 seconds before the first word appears. 37 seconds. So here's the question sitting in the middle of this video. Is a model you have to wait half a minute for actually usable? And notice
where that number lives. Not a footnote, not a buried issue. The fourth row of the benchmark table right under the headline, which is the part worth stopping on because it's rare than the
engineering. On a 24 GB M5 Pro, this engine does 31 to 35 tokens a second at about 2 GB of footprint. Strong numbers. Now read the row underneath them. Same machine, same model, running MLX, which
is Apple's own machine learning framework and the normal way to do this on a Mac. 76 to 82 tokens a second, roughly 2 and a half times faster. And that row sits in Turbo Fieldfare's own
benchmark file. Notice how unusual that is. A project whose entire pitch is a memory win publishes the speed comparison it loses by a factor of two and a half on the same page as its
headline with the winner's version numbers listed so you can go and repeat it yourself. But the number that traveled wasn't the speed one. It was the memory one. There's a popular video
about this project titled uses seven times less RAM. Seven times is what you get by dividing the biggest memory figure in that table by the smallest one. and the repo refuses that ratio in
writing. Its words, those counters measure different things and should not be compared as a direct memory ratio. That's graphics allocation. Process memory physical footprint. Three
different rulers under one headline. The sharpest objection came from a commenter and it's five words long. Unused RAM is wasted RAM. Mac OS fills memory you aren't using with a disc cache. So a
share of those expert reads never reach the SSD at all. They come back from RAM. The process doesn't count. So the 2 GB is real and it also isn't the whole bill. The author agreed when someone
pushed on it, take his M5 into memory pressure and decode drops from 35 tokens a second down to 27 on the same machine running the same code. And across the community reports, the spread is wider
still. three tokens a second on an 8 gigabyte M1 Air, 25 on a desktop Mac, one engine, one set of weights, and the result decided almost entirely by the machine you happen to own and how much
of it is free. So that's the engineering, and that's the caveat that comes with it. What's left is the why. Step back and ask why anyone would spend weeks building this. And the answer is
the first line of the readme. It's by Andre Mihilof, an iOS and metal engineer in London. And it reads, "Memory got expensive, so I gave a 26 billion parameter model a 2 GB budget." He isn't
being poetic. Earlier this year, contract prices for ordinary memory chips rose by more than 90% in a single quarter because AI data centers are buying the supply faster than the fabs
can make it. Memory is now the component the whole industry is fighting over, which reaches you as one line on a configuration page. Apple charges $400 to double the memory in a MacBook
soldered in at the factory, so the decision is permanent on the day you buy it. The SSD sitting beside it costs pennies per gigabyte. So, strip the engineering away, and this project is an
arbitrage. It moves nearly 13 GB of a model off the expensive component you can't change and onto the cheap one you already paid for. All the engineering exists to make that move survivable, and
it isn't one repo. Four engines ship this same idea inside 4 months. a one-day proof in March, a serious one in May from the person who wrote Reddus aimed at high-end Macs, a pure C version
in July that already has 25,000 stars. Turbo Field Fair arrived 2 weeks after that one, pointed at the opposite end of the shelf, the cheapest Mac in the room. More than 3 weeks after the thread, it's
still shipping. Five releases in the last 12 days, plus benchmark reports from strangers laptops. So, here's where I land on raw speed. Emillex wins, and it isn't close. If your Mac has the
memory free, use EmoLex. I'm not hedging on that. It's the recommendation. And the project's own benchmark file says the same thing. But on an 8 or 16 GB Mac with no upgrade path, which is most
people watching this, IMLex won't run this model at all. And Turbo Field Farwell. Where it fails is prompts. 37 seconds to first token kills any workflow built on pasting a document in.
I'd still install it. And not for the speed, for the table. Because the villain around here is a number published without the row that cost you something. And this is one developer
printing every row that hurts. So, if the ceiling on a local model is now disc reads and whatever RAM your machine happens to have spare, what exactly are you buying with a $400 memory upgrade?