RTX 5090 Is Ending? Don’t Panic-Buy a GPU for Local AI

0:00 The RTX 5090 on your autumn local AI short list sells for $2,400 to well over $5,000 depending on who you ask against a $1,999 launch price. Now, two leakers on X are

0:13 saying Nvidia is ending it, and that sounds like a reason to buy before it's gone. Your short list probably also holds a used 3090, a 5090 laptop, a Mac, and a

0:23 $150 old card. None of their spec sheets says which number decides what they can do for local AI. Two numbers do, memory capacity and memory bandwidth, and the rumor touches neither of them.

0:35 The report comes from WCCFTech quoting two leakers on X, Z Wang and Hong Xing 2020, and WCCFTech itself adds there's no official confirmation. Tom's Hardware says Nvidia didn't respond to

0:48 its request for comment. So, the case in this report for ending a $1,999 product is a couple of accounts on X and a notice from a factory with no name attached. XDA's January advice is that a

1:01 used 3090 is the value king for local AI because 24 GB of memory is hard to find, and that part is true. Beyond a model of 32 billion parameters, meaning 32 billion stored numbers

1:13 squeezed to 4 bits each, the 3090 runs out of room. By the end, you'll be able to read a spec sheet or the llama.cpp scoreboard and say which of those five suits your model.

1:24 Carry one question along the way. Which of them can even load a 70 billion parameter model at 4 bits since the card in the headline can't? You'd expect the fastest card to be the safe buy, and the

1:34 shared scoreboard seems to agree since on the same test the 5090 generates 290 tokens a second, a token being a word-size chunk of text, while the 3090 generates 158. Volunteers run

1:46 llama.cpp's public benchmark on their own machines using a 4-bit 7 billion Llama 2 file of about 3.8 GB. It times two things, how fast the card reads your prompt, called prompt processing, and

1:59 how fast it writes the answer. Because the rows come from different builds and drivers, they're not a controlled lab. It's a 7 billion model from 2023, which in local AI years makes it a family

2:10 heirloom. Only the bandwidth and file size logic coming up carries over from it, and it doesn't predict how a current 32 billion or mixture of experts model will run. But the same table has the

2:21 5090 laptop at 156.5 tokens a second, a near tie with the desktop 3090 at 158.2, even though its name says 50 series against 30. Prompt processing flips the

2:33 order with the laptop at 6,667 against 5,175. So, the same two cards win different tests. Nvidia's spec sheet gives the clue away,

2:44 listing the laptop chip at 896 GB per second of memory bandwidth, which is exactly half the desktop 5090's 1,792. Its generation score of 156 is a little over half of the desktop's 290. Shopping

2:59 by the model number is the trap here, because the laptop part carries 24 GB of GDDR7 against a desktop's 32 and half the bandwidth. So, for writing it behaves like a 3090, and for reading

3:09 prompts it beats one. When you send a prompt, the card first reads the whole thing in one batch. Batching reuses each weight across many tokens and keeps the compute units busy. So, this phase is

3:20 limited by compute. After that, it writes one token at a time. Every new token needs all the active weights pulled from memory again, which makes that phase limited by memory bandwidth.

3:29 Think of a workshop where the tool room is memory capacity, which decides whether a job's tools fit at all. The hallway to the workbench is bandwidth, which decides how fast the

3:39 tools arrive. Writing just one token means carrying the job's whole kit down that hallway once. Where the workshop picture breaks is the bench, since it says nothing about how fast the compute

3:49 works, which is is the first phase waits on. So, pasting a long document makes you wait on compute while long answer is paced by bandwidth. Apple's own MLX research gives the cleanest test of that

3:59 split. Moving from the M4 to the M5 raised memory bandwidth from 120 to 153 GB per second, about 28% and token generation got 19 to 27% faster. Prompt processing jumped by up to four

4:14 times, which Apple credits to the new matrix hardware. Each generated token reads the whole 3.8 GB file once, which gives you a ceiling a calculator can find. Bandwidth divided by file size.

4:26 For the 5090 that's 1,792 over 3.82, which works out to about 469 tokens a second against 290 measured or 62% of the ceiling. Run the same division on the 3090, the laptop, and

4:41 the M5 Max, and they all land between roughly 64 and 74% of their ceiling. The 3090's 936 GB per second is itself worked out from its bus width and memory speed.

4:53 The older M4 Max rated at 546 GB per second and scoring 83 sits lower at 58%, so the band is about 60 to 75 with that one as the low outlier. No primary source explains the missing third or so,

5:07 and overheads and imperfect memory use are only a guess. So, the spec sheet bandwidth is the brochure number and the scoreboard says to plan on about 2/3 of it. A second

5:17 artifact shows the same split the Apple silicon table in llama.cpp's GitHub discussions. An M5 Max emission from March scored 987 on prompt processing and 103 on

5:29 generation. And a June submission with a later build scored 3,347 and 119. Prompt processing more than tripled while generation rose about 16%, which fits the compute and bandwidth

5:41 split, though the two machines had different memory sizes. Compare the 5090 and the 3090 directly. Its bandwidth is 1.91 times larger and its generation speed was 1.83 times

5:52 higher. So, for writing, the extra money buys almost exactly the extra bandwidth. Prompt processing is where the gap widens, 14,073 against 5,175. Call it 2.7 times. Speed assumes the

6:06 model loads, and the second number decides that. Quantization stores each weight in fewer bits, so the 70 billion Lama 3.3 shrinks from 75 GB at 8 bits to 42.5 at 4. Since every token reads the

6:20 file, the smaller file also raises the speed ceiling. Qwen 3 at 32 billion parameters shows the same effect at smaller scale, with 19.8 GB at 4 bits, 23.2 at 5, 26.9 at 6, and 34.8 at 8. A

6:35 24 GB card holds the 4-bit file, and by subtraction, leaves about 4 GB for the context cache, the memory that holds the conversation so far. So, it's snug. A 32 GB card reaches the 6-bit file, while

6:49 the 8-bit one at 34.8 fits neither card. Now, take the 70 billion model, whose 4-bit file is 42.5 GB, bigger than the 5090's 32 and the 3090's 24. And even the 3-bit file at 34.3 misses 32.

7:06 Among the sizes that Page lists, only the 2-bit files fit, the 26.4 GB one on a 5090 and the 21.1 GB one on a 3090 with almost no room for context. How much answer quality 2-bit

7:20 compression costs is a separate question, so test it on your own prompts before buying around it. Here, the Mac stops looking like an oddball. Apple lists the M5 Max with up to 128 GB of

7:31 unified memory, one pool the processor and graphics share. So, a 42.5 GB file fits on the 64 and 128 GB machines. At up to 614 GB per second, dividing by the file puts the ceiling near 14 tokens

7:46 a second, a calculation and not a measurement. You'd expect two files of similar size to write at similar speed, and on a 5090 they don't. Hardware Unboxed measured Qwen 3's 30 billion

7:57 mixture of experts model at 234 tokens a second, and the dense 32 billion model at 61, both at 4-bit class quantization with a 4K context. A mixture of experts model keeps all of its weights in

8:10 memory, but routes each token through only a few of them. An Apple's test include a 30 billion model with 3 billion active, so the ceiling follows the active bytes, while the capacity

8:20 need follows the whole file. That's the workshop picture bending, since few drawers open per job, but the room still has to hold them all. Compared directly, it's nearly four times the speed from a

8:30 similar size file, though it comes from one independent site's test in a different quantization format from the dense rows. The 5090 laptop is the clearest trade.

8:39 VideoCardz list 24 GB of GDDR7 at 896 GB per second, so it has a 3090's capacity and a 3090's writing speed. Its 6,667 prompt score also means a shorter wait before the first word than the 3090's

8:55 5,175. On the 7 billion test, the M5 Max writes more slowly, 103 to 119 tokens a second against the laptop's 156, because 614 GB per second is below 896.

9:11 In exchange, it reaches up to 128 GB, a bit over five times the laptop's 24 by simple division. At the cheap end, Arya Hedayat's February post puts a GTX 1080 Ti about

9:24 $150 used with 11 GB at 62 tokens a second on the same test. The scoreboard lists a 2080 Ti at 107, and 62 tokens a second is faster than most people read, as long as the model fits in 11 GB.

9:39 So, the 1080 Ti and the 5090 both failed to open the 70 billion file, which is a kind of fairness. Back to the rumor. Tom's Hardware's headline expects a supply drought, while

9:49 the coverage adds that existing inventory would likely keep selling. The PC enthusiast says a 24 GB 5080 is targeted for the first quarter of 2027, with partners given only verbal notice.

10:01 Even that rumored 24 GB 5080 has no published bandwidth yet. So, capacity alone says only that it would load what a 3090 loads. Prices are where the sources disagree loudest, with one round

10:13 up putting the 5090 at $2,400 to $4,000 in the US and Tbreak at $5,000 to $9,600. That makes the $1,999 launch price read like a historical

10:26 document. A used 3090 runs from about $800 in XDA's January piece to $1,450 in October trackers. The rumor is one input, and it moves only the price and availability of a

10:39 single card. Next to it sits the file size of the model you want to run, a number you can look up in a minute. Which of these machines are even in the conversation depends on that number.

10:49 Start with that file size. If it's larger than the card's memory minus a few GB for context, stop there because a model that doesn't fit in memory won't run at these speeds. If it fits, divide

10:59 the bandwidth by the file size and keep roughly 60 to 75% of the result, which is where the dense model rows landed. For a mixture of experts model, dividing by the active weights gives a higher

11:10 ceiling, but the 60 to 75% band comes from dense models only. And the one measured example here is 234 tokens a second, so use a ceiling as an upper bound and still check the whole file

11:22 against memory. Try a 4-bit 32 billion model at about 20 GB. A 3090 gives 936 over 20, about 47. So, by that rule, you'd see roughly 28 to 35 tokens a second. A 5090 gives 1,792

11:38 over 20, about 90, which makes a band of about 54 to 67. The measured 61 for the dense 32 billion can three lands inside it, assuming its file is near 20 GB. So, call it a consistency check and not a

11:51 proof. If you run models up to about 32 billion parameters and want the fastest option, the better pick is the 5090. Its 32 GB holds a 6-bit 32 billion file,

12:03 and its 1,792 GB per second wrote 1.8 times faster than the 3090 in the shared test. The rumor is a reason to check prices this week, since existing inventory would

12:14 likely keep selling. And whether the premium is worth it depends on the price where you live. On a tighter budget with the same model sizes, the better pick is a used 3090, which gives 24 GB and a

12:25 little over half the 5090's writing speed, and its price decides it. So, check both cards where you live, because the quotes for the 5090 differ by thousands.

12:34 For 70 billion models at 4 bits, no single GPU on this list holds the file. The better pick is a Mac with 64 or 128 GB, where the M5 Max has a ceiling near 14 tokens a second on paper, which at

12:47 about 2/3 is closer to nine or 10. And that's a calculation, not a measurement. Take the 128 GB one if you want room for context, since the 64 GB one leaves roughly 21 GB for the system and the

13:00 cache together. If you want a laptop, the 5090 laptop gives you desktop 3090 speed up to 24 GB of model. The M5 Max gives up roughly a quarter to a third of that speed for up to 128 GB

13:14 of room, judging by those rows. As for a $150 1080 Ti, it's a fair way to learn on small models and a dead end past 11 GB. If Nvidia does confirm the end, expect the price quotes to move first

13:28 because the capacity and bandwidth figures on your short list stay exactly where they are. The thing that can change your verdict is your model's file size.

13:36 Back at the opening dilemma, the card in the headline can't load the 70 billion file at 4 bits, and neither can the 3090 or the laptop. What's the biggest model you actually want to run at home, and

13:47 does its file fit in 32 GB?