Is Local AI Now More Expensive Than Claude?

0:00 Run the napkin math on your own graphics card and it looks almost like a tie. A 450 watt PC burns about 47 cents of electricity per million tokens and Claude Haiku 5.5 which launched on

0:12 October 7th lists 50 cents per million. If you've been told that local AI saves you money, that 3cent margin makes it fair to wonder whether the card under your desk just became the pricey option.

0:23 Checking that claim comes first, and the arithmetic mostly survives. So the interesting part is what each of those two numbers count. Flavio Copess offers the sober version having priced a 4090

0:34 over 3 years and concluded that local rarely wins on cost alone which is a fair summary. Napkin math cracks in the one place it can't see. Electricity per token isn't a single number and worked

0:45 out from the published speeds it runs from about 2 cents per million tokens to 87 cents. What makes it swing that far? By the end, you'll know what Haiku's 50 cents really buys, what Local's 47

0:56 leaves out, and who Local still wins for. Start with the 47 cents because that arithmetic deserves a fair hearing. A third party Llama CPP benchmark from Context Studios, published in August,

1:07 has Quinn 3.8, a 27 billion parameter open model, decoding at 46.2 tokens per second on an RTX 4090 using a 4-bit version that takes 17.1 GB. Decoding is a generation step, one token at a time,

1:23 and it's the part you sit and wait for. A million tokens at 46.2 a second divides out to about 21,600 seconds, which is 6 hours of the card running flat out. 6 hours at 450 watts is 2.7

1:36 kwatt hours. At the 18.31 cents per kilowatt hour that the US Energy Information Administration reports as the residential average for July, that comes to about 49 1.5 cents. Working

1:47 backward, the claims 47 cents implies a rate near 17.4 cents, close to last Julyy's 17.45. So, the maths is sound. Both inputs are soft, though. The 46.2 is a third party benchmark, and 450

2:01 watts is the card's rated power with no reading taken while this model ran. Generating tokens mostly weights on memory rather than raw arithmetic. So, 450 is probably a ceiling. Inventive HQ

2:12 measured an RTX 5060Ti on a 7 billion parameter model at 81.2 tokens per second with a 180 watt cap and 80.4 at 150 watts with the wall reading 172.7 and 147.2 watts. So a 17% power cut cost

2:30 under 1% of speed. Different card and different model. So treat it as the general pattern with this PC's draw still unmeasured. It points the right way though and it's the one correction

2:40 so far that favors local. The real wall draw could come in under 450 and the electricity share would shrink with it. Now the other number because the 50 cents on the hosted side is the output

2:51 price only. Anthropic docs list input at 10 cents per million tokens and cash reads the price for text haiku has already stored at 1 cent with half off both input and output through the batch

3:03 API where you wait for results in exchange for the discount. So, the comparison lined up locals generation step against one line of that price list. Haiku 5.5 also has a long prompt

3:13 tier past 100,000 tokens and a context window of a million tokens. So, the long prompt side of the comparison gets its own turn later. Local looks best on the input side. A 4090 prefills, meaning it

3:25 reads your prompt in one pass at about 2,963 tokens per second. So, a million input tokens takes roughly 340 seconds and about 0.04 4 kwatt hours. Reading a

3:37 million tokens cost the card under a cent, so reading is nearly free, and the writing is where the meter spins. Divide 2,963x 46.2, and prefill comes out about 64

3:48 times faster than decoding. Prefill gets there by handling the whole prompt together, while decoding has to produce one token per step, each depending on the last. Set that against Haiku's 10

3:58 cents per million for input, and local wins the input line by more than 10 times on electricity alone. Hosted output roughly ties with local power, while hosted input sits far above it,

4:08 which favors prompt heavy work for someone who already owns the card. Anthropics Post says Haiku 5.5 costs about 75% less to run than Haiku 4.5. And the docs add that the new tokenizer

4:20 counts about 30% more tokens for the same text. So the discount shrinks depending on whose tokens you count. From here on, the doc's prices are the ones used. Electricity turns out to be

4:31 the smaller local cost. At launch, the RTX490 cost $1,599, and the Price Tracker GPU Poet lists its lowest average street price for September at $2,776

4:44 with October listings starting at $3,217. That's a price tracker aggregate with no retailer quote behind it. And either way, the card now costs more than it did

4:53 at launch, which is an unusual way for a graphics card to age. Take the September figure and spread it over 3 years and call it about $253 a day. A 4090 running flat out at 46 tokens per second makes

5:06 roughly 4 million tokens a day. Divide the day's cost by the day's tokens and the hardware comes to about 63 cents per million on top of the 49 cents of power. Flat out means 24 hours a day though,

5:18 which is a lot to ask of a card. At 2 hours of generation a day, the same card makes about 330,000 tokens. and dividing the daily cost by that gives close to $760 per million for hardware alone.

5:31 With power, you're near $8 against Haiku's 50. Copez's own numbers show the same split. He priced a 4090 at $1,800 with 500,000 tokens a day, which is 15 million a month. So his $59 a month

5:46 comes to call it nearly $4 per million. Only $860 of that is electricity, about 57 cents per million, which sits close to the napkin number. While amortization, spreading the card's price

5:58 over 3 years, supplies the other 50. Price per million also assumes the million tokens are worth the same, and the independent scores say otherwise. Artificial analysis rates the 27 billion

6:09 Quinn 3.8, the strongest small openweights model on its list, at 34 on its intelligence index. Haiku 5.5 lands at 43 at max effort, 41 at extra high, and 38 at high on the same index, second

6:23 of 180 on its page. Since it's a composite score, your own workload may not match it, but it's the closest sideby-side score available. Bigger open models score 44 to 51 in various

6:35 listings, but they need far more than a single consumer graphics card. So for 14090, the 34 is where the listings top out. On speed, artificial analysis lists Haiku 5.5 at about 240 tokens per second

6:48 against the 4090s 46. So it's roughly five times faster. A million tokens takes about 70 minutes hosted against 6 hours local. 6 hours for a million tokens is most of a workday, which is a

7:00 lot of waiting for a saving of a cent or three. Per token comparisons also skip how much each model talks. At max effort, Haiku 5.5 produced about 440 million output tokens across artificial

7:12 analysis's evaluation against a median near 100 million, while Quen needed about 200 million. Haiku writes more than twice what Quen does across the same evaluation, which eats into its

7:22 cheap token price. For a model named after a threeline poem, it has a lot to say. Context length is the next lever on the local side. In the same benchmark, the 4090s decoding slows from 46.2 2

7:34 tokens per second at 4,000 tokens of context to 38.4 at 64,000 which stretches a million tokens from 6 hours to a little over 7. Run that through the same 450 watts and 18.31 cents and the

7:49 electricity alone comes to about 60 which is now above Haiku's 50. Meanwhile, the faster RTX5090 does 74.8 at 4,000 tokens of context but only 26.2 at 64,000. So the same maths gives it

8:03 about 10 and a half hours and 87 cents per million. Why the 5090 falls so far at long context isn't explained in the benchmark and 450 watts is an assumption for that card too. So read 87 cents as

8:16 arithmetic on the same assumptions with no meter behind it. Long prompts cost more on the hosted side too. Past 100,000 tokens, Haiku's input goes to 50 and its output to $2.50, a five-fold

8:28 jump on output. The local benchmark stops at 64,000, so the two tiers don't line up past that, and the long prompts comparison stays open. Most of the corrections so far have pushed against

8:39 local. The card costs more, the model scores lower, the speed is a fifth, and long prompts drag electricity up. Going the other way, the biggest correction explains that swing from 2 cents to 87,

8:51 and it's how many requests the card serves at once. Generating a token waits on memory, as the 5060 Ti showed. Servers usually read the model through memory once per step. And when 64

9:02 requests share that step, they share the read. That's the general pattern. And no source here measures it on this setup. Picture a bus. The trip costs the same with one passenger or 64. So the riders

9:13 split the fair. The picture breaks on the number because the bus predicts a 64-fold drop in fair and the report shows about 25fold. Contact Studios reports an RTX3090 power limited to 250

9:26 watts serving about 1,000 tokens per second in aggregate across 64 parallel requests against 40 for a single stream. One community tuned guide with no control test is behind that figure. So

9:37 treat it as a strong hint. At 1,000 tokens per second, a million tokens takes about 17 minutes. Keep the same 450 watts for comparison, and that works out to 0.125 kwatt hours, around 2.3

9:51 cents, roughly 20 times less than the single stream's 49. Since this sets a 3090s batch result against a 4090 single stream, it estimates only the size of the effect. One person at one keyboard

10:03 has a smaller lever. Speculative decoding, which guesses several tokens ahead and checks them together, has a quoted 1.8 one-time speed up for a single user. If that carried over, 46

10:14 tokens per second becomes about 84, which would put power near 27 cents per million. The obvious objection is that your one person in one chat and a single chat can't fill 64 slots. That's right.

10:26 So, the 20 times belongs to batch work, like summarizing a pile of documents. The hosted side has a batch tier, too, at half price, which puts batch haiku output at 25. spread a $2,776

10:39 card over 1,000 tokens per second running all day and the hardware share drops to roughly three cents per million by that arithmetic. Using a 4090 price on a 3090 result is a stretch. So treat

10:50 this as an illustration that also assumes the card stays busy all day and keeping it busy is the hard part. Your electricity rate shifts all of this. Those 18.31 cents are a national average

11:02 and state rates run from about 12 cents to 46. At 12 cents, the single stream power is about 32 and at 46, it's about $1.24. So in an expensive state, the local side has to be busy to compete.

11:15 Put together, that gives two different ways of paying. Local is bought by the hour because the card costs the same whether it's generating or sitting idle, and a token share of that hour depends

11:24 on how busy you keep it. Haiku is bought by the token, so you pay nothing for the hours it sits unused. Anthropic's inference costs aren't documented. So its 50 cents is a price and its own

11:35 electricity bill stays a private matter. Locals 47 cents covers only the electricity part of your cost, which is why it can look cheaper while the card, the speed, and the quality sit outside

11:45 the sum. To check your own setup, divide a million by your tokens per second to get seconds and divide that by 3,600 for hours. Multiply the hours by your wall draw in kilowatts and then by your rate

11:57 per kilowatt hour and you have your electricity per million for the card. Divide its price by the millions of tokens it will actually generate in its life. Add the two, then put that next to

12:08 Haiku's 50 cents per million output tokens plus 10 cents for every million prompt tokens you send. For a single person chatting or coding with no GPU yet, the better pick is Haiku. Prompt

12:18 heavy work doesn't change that because by the same arithmetic the card alone comes to about 12 cents per million prompt tokens at two hours a day already above haiku's 10. Hosted is about five

12:29 times faster and scores 43 against 34. Local adds 63 cents per million of hardware if the card runs around the clock or about $760 at 2 hours a day. If you already own a 4090 class card, the

12:43 card is a sunk cost and the sums fall back to electricity, roughly 50 cents per million at short context and 60 or more at long ones. On money that's a tie, so the call turns on privacy,

12:54 offline use, and whether you can live with the speed and the score. If your work is batch with many requests at once, a 34 on the index is good enough and you can keep a card busy, local can

13:06 win. By the arithmetic here, that's on the order of 2 cents of power plus about three of hardware per million against 25 cents for batch haiku output. If that community report holds on your card for

13:17 that group, the napkin math undersells local resale value. The rest of the PC and idle draw all sit outside this math and they nudge the local total in different directions. So, the napkin

13:27 math was right about the electricity and left out the card, the speed, and the quality. For one slow stream on a card that's already paid for, 47 cents is a fair number. And Haiku's 50 cents is the

13:39 price of one line in a longer price list. If your own card spent a week on a batch job instead of waiting on a single chat, what would your cost per million tokens come out to? And does it beat 25?