Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Sonnet 5.5 is roughly half the cost and twice as fast as Opus 5.5, but the real decision between them comes down to task clarity: Sonnet excels when there's a clear spec and an objective definition of done, while Opus is better for vague, creative tasks where you need a thought partner. Across seven real-world tests, Sonnet won four and Opus won three, but the difference often came down to whether a second, more focused prompt on Sonnet could close the quality gap for far less money.
Key points
Sonnet 5.5 costs $2/$10 per million input/output tokens, half of Opus 5.5's $4/$20.
In a head-to-head landing page test, Opus's output was better overall but cost roughly double Sonnet's $13.
Opus produced a more polished and creative motion showreel than Sonnet, justifying its higher cost.
For a vague prompt requesting a 90-day action plan, Opus delivered a more useful, structured artifact despite higher cost.
When both models used a specific skill, their outputs were nearly identical, making Sonnet the cheaper choice.
tools
techniques
tags
confidence
importance_score
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
Cloud Sonnet 5.5 is here and it is a clear upgrade over Claude Sonnet 5. Running 30% faster as well as costing 30% less for most work. So, I've been playing around with it all day on my
actual real workflows. Things like building websites, taking a bunch of messy data and turning that into spreadsheets. Also, Excel sheets with different views as well as formulas
inside of every single cell, motion design and editing videos, interactive dashboards that have real-time data syncing, project management and planning, and so many other things as
well. But specifically, I've been testing Son of 5.5 against Opus 5.5, so you know exactly when you should be using which model. Now, in all these comparisons, I'll go over how fast they
are, the input and output tokens, and how much they actually cost you in dollars. So, let's not waste any time and just get straight into this one. Okay, so before we jump into all of the
different demos, I just wanted to lay a foundation here, which is how much does Sonnet 5.5 actually cost versus Opus 5.5? The answer is roughly half. So here you can see Sonnet 5.5 is $2 per million
input tokens whereas Opus is four and Sonnet is $10 for the million output tokens whereas Opus 5.5 is 20. So Opus is roughly double the cost of Sonnet 5.5. So we're here to see if Opus is
double the quality of Sonet 5.5 and where to use each model because Claude actually came out themselves and dropped this little guide that said, "Hey, here's when you should use each model.
If you have something well scoped for everyday coding, fixing bugs, sonnet it high volume everyday development, sonnet. polished documents, slides, spreadsheets, one pagers, sonnet. You
should basically be using Opus if it's complex work requiring careful judgment, including long horizon, agentic coding, and knowledge work. And for your hardest problems where you need the most
intelligence, opus 5.5. I really like this section down here. Sonnet 5.5 fits best when the task has a clear spec and a way to check the result. And so, the way that I interpret this is basically
if you have a task with an objective definition of done, then try that out with Sonnet 5.5 first. If you need some more creativity and you're looking for a thought partner to help you decide what
the definition of done is and it's a bit more subjective, then that's where I would use opus as my model of choice for that task. And then what I'd do is I'd run different skills with different
models and I'd see, okay, for this skill, Sonnet does better. For this skill, Opus does better. So in the future when I want to run this skill again, I know which model to use.
Anyways, you guys know the drill here. I gave all of these different sessions the exact same prompt. And now we will be reviewing the results. I did Sonnet 5.5 and Opus 5.5 both on high. When it comes
to the actual effort level, they both used high. So the first thing was building a engaging and immersive but high converting landing page for perk form. And I said that they could use
key.ai to generate anything they need. And you can see I had to correct it key.ai, not G.AI. So I should actually probably add that as a misspelling inside of my glido to correct this to
key. Okay, cool. All right, so let's take a look at these two outputs. I'm not going to tell you which one was which until the end. I really like how when this one loads up, we have that
text come in. Really cool. We can see that this can is interactive. It syncs to what my mouse is doing. Right off the bat, I noticed that this can is kind of a 3D rendering. So, it's not actually
the exact can like pixel for pixel the way it should be. So, it had to kind of invent that. I don't love that. But, I do like the effect that it gave for us. I also like how in the back you can see
Oh, wow. Actually, if I click and drag it like scrolls the can, but you can see in the background we can see like perk form. We can see text in the back, which I thought was pretty cool. Anyways,
let's go ahead and scroll down. We can see the different flavors here. We can actually switch right from the hero section, which is pretty neat. As we scroll down, we get the can flooding
away. And then we have two drinks, coffee, and protein coming into one, which gives us our one can, which is our perk form. We can see we scroll down into a little section where we have a
gym bag, phone, keys, and as we scroll, we have the time going up, and now we grab our perk form. We have a little flavor section. So, we have the three different flavors. I'm able to see that
I can click between them here. So, we have our salted caramel, we have our vanilla latte, we have our bold mocha. It's sort of starting to tell a little bit of a story here. And then that is
basically the end. And we have these three dynamic cans coming back into view. Okay. So, that was one output. Here is the other one. So, right off the bat, this one isn't as like boom right
in your face. It's not as much wow factor, I'd say. We do have sort of an element where it's syncing to the mouse. You can see as I'm moving, you can see these are on different layers. Also, the
key is on a different layer over here. And I think that honestly this looks a little bit cheaper when we look at just kind of like the hero section. But as we scroll, we can see that those other two
drinks kind of come down here. It's doing a similar style where we're like scrolling through a time. And I don't love this one as much. It's kind of making all of your eyes and all of your
attention go to one spot. So it's definitely telling a different style of story here. Even though all the branding and everything else is consistent, this is using the actual real generated image
inside of the brand guidelines for the product. So, that is better that it gave us the actual um I guess image in a real way rather than back here if you guys remember it was like a generated
version. Here's a cool little animation here. We zoom into that image. All of these were different independently generated images. I think that this looks really nice. Ultimately, this is
the better output in my mind. The one thing I don't like as much is this section. I think that this version tells a little bit of a better story, but ultimately this is the better output.
Even though I don't like the hero section as much, this is the better output. So, this one was Opus 5.5 and this one was Sonet. I think Sonic killed the hero section. If this would have
been a real CAN and then Opus 5.5 missed that, but everything else of Opus' version I think was better. So, if we look at the stats here, we can see that Sonnet ran for 28 minutes whereas Opus
ran for 52 minutes. Sonnet cost 20 million input tokens, 134K output tokens. Opus was 37 million and 148k output. and Opus cost about double here. So Opus ran for about twice as long and
cost about twice as much, which is pretty consistent with what we saw when we actually look at the pricing. The question to me now would be, is Opus' output two times better? I would say no.
So ultimately, what I would probably want to do here is I would reprompt Sonnet and say, "Hey, this is really good, but don't use like this fake image. Use our actual real images." And
I bet that it could swap everything out. And then I bet that it could give us the level of output we were looking for with a real can and it wouldn't have costed us $13. So ultimately that's why I think
that we need to give this run, we give the win here to Sonnet. But either way, this stuff is subjective. So take it as you will. But that is what we're going to do right now is I'm going to give
this first output to Sonnet. And in my latest videos, you guys have been saying like, "Oh, please can you use the subscription um usage instead of the API billing?" No, I think this is a better
way to show the actual cost. These costs are very much correlated to your subscription. So, just think about it like that. But the reason I'm not going to do subscription is because I don't
know if you're on the 20 bucks a month plan or the 200 bucks a month plan. I also like to run all of these in parallel and I don't want to like sit and wait 52 minutes and then jot down
what percent it was and then do this one for 28 minutes and then jot like, you know, I'm just gonna run this in parallel. Anyways, let's move on to number two. Real quick, guys, I've got
this completely free website design skill that I'm giving away. I use this for every single website that I build, including our AI Automation Society site, which I think is pretty slick, and
I absolutely love this website. It's called ScrollCraft, and it understands things like scroll driven animations, and layering, but also it has things in there like taste, typography, spacing,
depth. It's just a really good website design skill in general, and it's completely free. So, the link for this will be down in the description, but let's get back to the video. All right,
for number two, what I did is I said, "Hey, I need you to make me a motion showre skill about glido." and you do the research, make sure that all of this is consistent in branding, typography,
you know, all of this kind of stuff. You are the motion designer here. So, just impress me. So, let's first look at Sonnet's output. Refactor this function to use uh async
await instead of promises. I will fold those into the next revision. Reply saying it looks good and we can start next week. We try logic to the API client.
Okay, really not bad at all. I thought there was a lot of creativity here where it's kind of showing off what the product actually looks like when you use it, which I thought was really nice. The
voice over that it did, it generated that with maybe 11 Labs or something. I also really liked how these pixels here formed a G and that actually turned into an AI generated video that it made. So,
I thought that was pretty nice. Overall, I was honestly not expecting a Sonnet model to do this good with this motion show reel. Now, let's go take a look at Opus' output.
Refactor this function to use async await instead of promises. Okay. I mean, I don't think you can beat that. Opus 5.5 is just so good as of now at motion design. It just felt more
glido. the background, the different animations it made, it just felt like even the copy was better. It also went to our website and pulled this quote from me that was on the site, which was
very fast. But right here, this was actually the quote from the website. So ultimately, this output much better from Opus. Now, when we look at the cost here, Opus was faster and it was also
less output tokens, but ultimately it was more expensive. So, Sonnet was $14.37 whereas Opus was $2055. I think though we're gonna give this one to Opus. If I'm still looking through
the lens of if we had Sonnet try again to get up to $20.5, I still think Opus' would be a better output. It just had a better taste with like motion and the sound design. And
sound design is something that I think is really hard to get right even as a human sometimes. And the fact that Opus is doing this so consistently, very good. So, I'm giving this one to Opus.
Okay, so for number three, I said very, very vague. create me a researchbacked 90-day action plan for getting how they AI to a thousand subscribers. So, let's go first take a look at Sonnet's output.
Okay, so one thing to call out here is both of these models originally when I shot off that prompt, they just gave me a markdown file. And then for both of them, I gave them the exact same second
prompt, which was turn that into some sort of document that I can view that's easier for me to read. And this is Sonnet's version. It created me an HTML file and it gave it to me locally. And
then Opus did the same thing except for it put it as a clawed artifact. So, that's one little tiny difference, but besides that, this is Sonnet's output. Let's see. 1,000 subscribers in 90 days.
It also threw our logo up here, AIS Media, September 28th. We have day one launch as well as the day 90 goal. It's interesting that it chose to launch this on November 2nd rather than like next
month or something, Q4. Anyways, we have a timeline here. So, we have episodes on the bottom. We have subscriber gates and getting to a,000 by day 90. We can see that we have some other things here like
promotions. Test send full list full list. We have Nate promotion sends out six of them. Community posts five of them. 500 subs. Holidays locked. Interesting. We have different phases.
So what to do from October to November and then two weeks on launch and then a month in the next stage and then compounding. So it's showing us the different sort of milestones to hit in
these different phases. It's showing us the different sources that we can get subscribers from. None of this is interactive. doesn't look like. It's just showing us what this could look
like. We can also see that we got some scroll animations coming in there, which I thought was a nice touch. As you can see, the text comes in. It's also looking at shorts and LinkedIn and guest
kit and scorecards and other decisions like that. Okay, so this is good. I wouldn't say that this is like super super helpful for me at the moment, but not too bad. And here we have the
version with also scroll animations from Opus as a cloud artifact. This one chose to start October 1st to December 29th. So, it's actually like a Q4 90-day sprint, which I think is a bit more
aligned based on like what's going on in our business right now. So, now we have the 90 days. This one gave us a little bit of a different sort of view, which I actually like better. I'd say we can see
when we're going to launch. And from now until launch, we'd be kind of setting up auditions and we'd be doing guest applications and we'd be, you know, announcing all of this kind of stuff,
recording these episodes. we'd be looking at getting out shorts and hitting the email list and community posts and all this kind of stuff. I think that this view is a little bit
easier to digest than what's going on here. They're just a bit too much going on here and it's not as clear. Whereas this seems a little bit more clear to me. We have different phases here, but
this one's basically each month, which I think is better. Build, launch, compound. Same thing. Where do these actual subscribers come from? The list, YouTube organic, Nate's channel, things
like that. Here are things that we actually need to decide on. And then we can open the full plan which actually takes us to oh wow this takes us to like an actual very specific write up with
like tasks. So this is basically something that I could throw into my OTAA now. And we have these actual tasks that we can check off. Okay, I think this one's pretty clear. Opus wins this
test. Now when we take a look at the stats, Opus ran for 20 minutes. Sonnet ran for 14. Sonnet cost a little under $5, whereas Opus costs a little over $9. So, if we had Sonnet take another stab
at it, where would Sonnet be? Now, this one gets really interesting to me because if you think about the actual prompt, I didn't really give it guidelines at all. I said, "Create me a
researchbacked 90-day action plan." That was it. And if we think about what I said earlier, if you have an objective definition of done, use Sonnet. If you need more creativity and you're looking
for a thought partner, use Opus. And this is definitely an example of me looking for a thought partner and looking for a model to just take a ambitious goal or sort of a vague goal
and just run with it. I think that if I would have given both of these models a very very specific prompt on here's what I want, here's how it should look, here's blah blah blah, I think that opus
and sonnet would have given me a similar result but sonnets would have been cheaper. So I do really think that that's interesting to think about the way that you talk to these different
large language models. But in this case I can't ignore how much better opus did with this vague prompt even with the cost. So, this one is going to go to Opus as well. So, right now we're at
Opus is winning two to one. And really, guys, I'm not trying to come in here and say like which model is better than which because ultimately Opus is a better model, but I'm just trying to
hopefully show you guys like now you might have a better idea of when to use each model and or how to use each model. So, let's hop into number four. All right. So, in this one, I gave them a
bunch of company data, mock company data, and I wanted to get an Excel sheet as well as a investor pitch deck. So, we're going to do Opus first. Here is the deck that Opus built. It generated
this image in the back. We've got 18 slides. We've got generated images here as well. We have the problem. We have the product. We have who we serve. We have some traction, so some statistics
in here based on the mock data. We've got the business model, customers. Now, this doesn't look too bad. We've got visuals. We've got, you know, not too much going on on each slide, but I will
say this is the infamous clawed design font with the Fs like that and the numbers. It doesn't look super premium in my mind. Like this doesn't feel as premium as I'd like it to, but either
way, the slides aren't designed super super poorly or anything. So, that is the output. I mean, this really isn't bad at all from Opus 5.5. And then on the Excel sheet, we have a dashboard,
inputs, monthly metrics, we have different views. The dashboard is really nice with all these stats and all of these different visualizations in here. We have the different inputs. We have
metrics, quarterly, P&L. What I really like is in a lot of these cells, we see formulas. We see formulas basically everywhere. Which means if we had to come in here and change up the data, the
whole sheet would update. I don't like when AI models will give you a Excel sheet with no formulas and now you have to basically it's all static. It's all hard-coded data and if you needed to
make changes, you would have to go ask the agent to make those changes for you. I hate when it does that. So, it's really good that Opus is putting I mean there's tons and tons of slides here,
like a ton of slides or sheets, I guess, tabs. What I like is that they have formulas inside. So, it's really not bad at all from Opus. We've even got some color coding and stuff. Very good Excel
sheet. This is interesting because if you guys remember earlier, Opus made an artifact and Sonnet didn't. But in this time, Sonnet made an artifact. So, let's see how this one's looking. This is
Bright Path Analytics. We see our intro right there. We have I believe this is the exact same font that Opus was using. So anyways, we have um company snapshot, the problem, the product, who we serve,
business model, traction. I mean, this is very very similar. Like there's not really any difference in my mind between the two different decks. They feel and look very very similar and sonnet
actually gave us more slides. Now, let's open up the Excel sheet we got from Sonnet. Immediately, it doesn't look as premium, but we go into the dashboard. The dashboard looks just fine. We have a
lot of visualizations here. monthly data. We can see that we have lots of formulas in these calculation cells. Same thing with the P&L. We have some SAS metrics. We have assumptions. We
have organic forecast. All of these are different lookups and formulas as well. Wow. Lots of different tabs here. Pipeline, customers, marketing, headcount. So, this one honestly might
have given us like too much. I don't know. If this was real data and this was my real company, I'd be able to tell you guys better like how useful all this is. But ultimately, I wouldn't really be
able to come in here and say that one of these outputs doing this sort of like document creation was significantly better than one of the other AI models. So, I really think in this case,
whichever model was cheaper and faster is who I'm going to say won this round. And wow, in this case, Opus was actually cheaper and faster. So, we don't even have to discuss this one. Opus wins this
one. No question. Cheaper, faster, and the outputs were pretty much identical in my mind. Now, what if we had some skills built around it? That would be another interesting thing to test. But
in this case, I didn't utilize a skill. I just said, "Hey, make these things for me." All right. So, here's an example where it actually did use a skill. I have a skill called HTML explainer, and
I asked it to use that skill or I didn't ask it to, but it did. Where I said, "Hey, what models could I run on my current device? I don't know anything about this, so make it easy to
understand. Lots of pictures, blah, blah, blah." So, let's take a look at these outputs. All right, so this one is Opus. What AI can my PC run? You can see this is branded. It's just a super
simple, you know, HTML explainer style skill that I use sometimes. So, we can see we have graphics card, memory, processor, storage. We see the graphics card memory versus other things, which
is like a gaming PC at a data center chip. The 32 GB rule. The model has to fit in your graphics card. So, we probably couldn't run this 43 GB model, but we could run a 20 GB model. So, that
is a nice little simple rule. We can see that we also have shunk copies. We have speed, so tokens per second on our graphics card. What to install? Everyday chat. It gives us a suggestion. Coding
hard problems. Gemma 4 for fast replies. Super super cool. Too big for one PC. Start tonight. It tells us exactly what we would need to do. Okay, cool. Very simple. Not too bad at all. Okay. And
now we have the exact same skill, exact same prompt, but this is the sonnet version. So, local AI models your PC can run. It actually made us a little visual, which I thought was cool. 32 GB,
the number that decides what local means. Okay, so it starts off with what local means, which I don't think the other one did. The other one basically just went straight into RPC. So that's
good that it did that. It broke it down simply with another little visualization. Three places to keep model desk, shelf, warehouse. Okay. Fast, slow, storage only. What fits?
Green fits, amber crawls, red won't run. So it doesn't exactly tell me why because of the whole gigabytes thing, but it does show a nice little visualization and some other AI models.
So that's not too bad. It also shows us speed, pick by the job, similar to the last one, and how to actually try one out right now. So ultimately, if we really think about how would we want to
understand how this stuff works, I do think that Opus' output was better. Like this one is more visual, I'd say, but it doesn't actually explain it to me as well as the Opus one did, unfortunately.
So I would say that Opus wins this output. But if we think about this cost here, similar timing, Sonnet cost half of what Opus cost. So, if I prompted Sonnet one more time and said, "Hey, I
am still confused about this, this, and this. Can you just add some more info here?" I bet that Sonnet's output would be better. And I would also argue that I think that Sonnet's output visually is
better. And that's one of the the pieces of this skill, which is a very, very simple skill. It's not refined at all, but ultimately, I think that if we prompted Sonnet again, we would get an
output better than what Opus gave us for $3. So, I'm going to give this one to Sonnet. Okay. So, in this next one, number six, I said, "Scrape X for all the news that's been going on today.
Give me an interactive dashboard so I can view what's going on and see what's coming out and just get prepped for all these different things going on in the AI space." So, let's take a look at
Sonnet's output. This is a local host that it created from me. We can see that we can see when this was last scraped. We can hit refresh. We can see how many posts were tracked. So, 386 posts
tracked from 676 scraped. We can see some headline news right up top. We can see what is trending. We can mark fits my channel or hide covered or pinned only. Okay, interesting. We can also see
the heat score. So, Claude Sonet 5.5 100 heat metastarts meta enterprise platform 94 heat. We can maybe click into this stuff. Nope, I can just hover over. I can open details. Okay, cool. So, I can
actually open the actual post that it's sourcing, which is pretty nice. We can mark it covered. We can add to plan. So, now we get this list with like a content plan over there. Not too bad. We can
also sort by newest. We can sort by channel fit. We can look at the last 48 hours. We can see what's coming this week, dev day by OpenAI. We can see the calendar. Okay, this is not a bad view
at all. And now here we have Opus' version, which off the bat just kind of looks a little bit better to me. You can see that this one's still scraping X. As you can see, it's showing us a little
progress bar. So, it is working, but this view looks a little bit more visual. It's showing us posts per hour on AI. It's showing us how many posts are rising and cooling off. And this
view, in my opinion, is just a little bit better. I can also switch up here this week. Okay, nice little view here. We can see content plan, so I can add things to my list. We can see the raw
feed as well. So, that's pretty cool. I can see sort of a heat score as well. It's tagging things by rumor or shipped or discussion. And it's pulling up real posts on the side here. And let's see,
how do I add something to right there? I can click add to content plan. I can open the top post and it goes to X. Okay, very cool. So, I'm really starting to understand a pattern here, which is
what I said at the beginning. If you know exactly what you want, Sonnet is probably going to be able to do a good job for you. But if you need the creativity and you send an open-ended,
very vague goal, Opus is just going to handle it better. And that's become very, very clear to me. In this case, Opus ran for about 7 minutes longer and cost $6 more. And if we iterated four
more times with Sonnet, or as much as it took to get to $8 of output, but we had a clearer goal, then Sonnet would definitely give us that level of output. There's nothing in here that Sonnet
would have not been able to code or create for us. It's just that Sonnet doesn't think about that as much and it won't go kind of like outside the box as much as Opus will to give you a better
experience. It's basically just going to give you what it thinks you want and it's going to be the the baseline. So, if we're sticking with my rule here, because Sonnet was much cheaper, I'm
going to give this one to Sonnet. But ultimately, like right now on one prompt, Opus' output was better there and there's just no denying that. So, let's go on to the last one, number
seven. All right. So, in this one, I gave Opus and Sonnet a link to one of my YouTube videos and I said, "Create me a YouTube resource guide." So, this is using a skill once again. So, the skill
is defining what good looks like essentially. So, let's take a look at Sonnet's output. First, we have the AIS header up top. We have this font. We have Nate Herk linking to my YouTube
channel. It starts off with the challenge and the rules. It goes over the S&P benchmark. It goes over the actual steps of how I built this. We have the problem, the solution, day one
and day two results, day three, day four, day five, day six, day seven, final results, lessons learned, and we have key takeaways down here. They're pretty much the exact same length. This
one is like seven pages, but it's really like I don't know, there's a big gap here, whereas this one, very similar. Same link, same title, same header, the overview, the rules, the benchmark, how
the system was built, um day one, day two, final results, all this kind of stuff. It's very very similar, which is why this one would be essentially whichever one was cheaper is the one
that I would go with here. In this case, you can see that Sonnet was cheaper and faster about half the speed. So twice as fast and a dollar40 cheaper. So this one is going to go to
Sonnet because the outputs were not too different enough to actually beble to say, "Oh, one's really better than the other." And it's because in this case, I know we've seen some other examples
where it didn't always match, but in this case, I had a skill that I do use a lot. And a model like Sonnet that's less intelligent than Opus was able to just follow my instructions because it knew
what I wanted more clearly. So, this one goes to Sonnet. So, ultimately in this test, we had three, four go to Sonnet and three go to Opus. So, we did technically have Sonnet winning this
challenge, but I think you guys understand this wasn't really a head-to-head. It was more like how and when to use each model. And here are the final stats for you guys. You can see
across these seven sessions, Opus ran for about 46 minutes longer. It spent about 38 million input tokens more as well as actually less output tokens, which is interesting. But it did cost
about $16 more than Sonnet 5.5 in all these runs, which when you think about the fact that it's actually double the cost of Sonnet, it didn't cost double the cost in this experiment, which is
also interesting. I've talked a lot about how I see each of these models fitting into my workflow now. And hopefully you guys have a better idea of how this all should work for you. But
ultimately, you got to get in there and you got to test it on your own skills and your own prompts and your own processes. So, that's going to do it for this one. That was my experiment today.
I hope you guys enjoyed. I hope you learned something new. And if you did, please give it a like. It helps me out a ton. And as always, I appreciate you guys making it to the end of the video.
I'll see you on the next one. Thanks, everyone.