Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Claude Opus 5 is often cheaper than Fable 5 and can outperform it on coding and verification tasks, but Fable 5 remains stronger for creative and visual work. The choice between models depends heavily on the specific use case, prompting strategy, and desired output quality.
Key points
- Opus 5 is half the cost of Fable 5 per token but often uses more tokens, making total costs sometimes higher.
- Opus 5 outperformed Fable 5 on coding agent benchmarks like Frontier Bench and Cursor Bench.
- Opus 5 is stronger at verifying its own work and iterating until success, a key capability for AI agents.
- Fable 5 produced more visually appealing and creative outputs in tests like Excal diagrams and presentations.
- Model outputs are nondeterministic; the same prompt to the same model can yield drastically different results.
- Opus 5 is overkill for many daily knowledge work tasks; matching model intelligence to task complexity is important.
- Using Opus 5 as a delegator while reserving Fable 5 for orchestration can save session limits and costs.
Tools mentioned
Techniques
- verification loops
- sub-agent consensus
- model orchestration
- token efficiency analysis
- prompt engineering for agent behavior
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
All right, so Claude Opus 5 is here and if you start to look at the benchmarks, it's really interesting because it shows us that for a lot of things that I care about, it's actually better than Fable and it is half the cost of Fable. Ultimately, Fable 5 is still Anthropic's most impressive and, you know, strongest model. But for a lot of these things, you know, I've realized when I'm doing knowledge work and when I'm building, you know, my videos or my research or whatever it is, Opus is more than enough power than what I need. And when you look at some of these charts, it's really interesting because it shows on things like the Frontier Bench and the Cursor Bench and this coding agent index that Opus is actually outperforming Fable and it's cheaper. And this really shocked me.
So, obviously, I like to take all this stuff with a grain of salt. It's fun to look at and it's good to look at, but you want to actually get your hands dirty and run these models through your own actual workflows. So in today's video, I'm just going to break down a bunch of different experiments that I ran with Opus 5 versus Fable 5 and break down things like the cost, the time, and the tokens so that you can start to understand where you should work in these different models within your workflows. All right, so pretty much all of the experiments that I've been running today that I'm going to show you guys, I did within Claude Code, which means we're comparing the models, but also inside of the Cloud Code harness. and the variable is the same so it doesn't really change too much but I did do a few tests where I was actually in clawed chat and I was just you know seeing how they felt without a harness wrapped around and let me just show you one quick example so here I asked Fable 5 and Opus 5 to generate me an Excal diagram that accurately and visually explains how semantic search with vectorization works on a large data set for AI agents and it's interesting here because there's no skills that it can use and it doesn't have any context of me or you know any really way to verify all it did was it spit out a um JSON file of Excal for me and then I pasted it into Excal.
So, here's what we got. Fable came back with this version over here where we see we've got like our indexing pipeline. We have a large database and it looks like it actually misspelled this right here, which is interesting. Large. Oh, data set.
Okay, it was just like not expanded enough. Same thing over here. Vectorize. And this is part of the whole um it had no way to verify. And as you guys know, if you've been kind of building agent loops and stuff, verification is so so important.
So anyways, large data set, we chunk it up. We vectorize it with an embedding model. We then get our embeddings, which is just like the numerical representation of the data. We put it into a vector database here. And then we can actually start to search.
So we've got similarity and that's on, you know, points being close together. So your question lands here and it would grab the k nearest neighbors and different meaning is farther apart. We've got different clusters here. And then we come down here to the actual query. So if the user asks how I get my money back, the agent searches the knowledge base and then it does semantic search.
It looks it up with the actual query vectors and we get the matches back. So pretty accurate. I will say though, Opus' layout seems a bit more organized, right? Like it's it's got boxes and it's got I mean this might not be as visual. You could argue you could argue that Fables was more visual, which you know I think that that's true, but this definitely feels more organized.
it feels a little bit more detailed as well. So, that's just a very subjective exam. A lot of the stuff that I'm going to be talking about today is just really opinionated and subjective, but I'm going to still give you my own thoughts. So, in this example, I think that if I wanted to teach someone, I probably would take Opus 5's version here. Okay.
So, let's start off with the first test I ran, which was basically giving them a huge codebase and having them look through any bugs and looking through like the expected behavior and some instructions like that. So, it had to do some exploration here and help us out, right? So, I set the goal and I gave it this prompt. And then what I did is I had Codeex review the output that Fable 5 gave us and that Opus gave us. So, real quick before we look at the results, Fable took about 11 minutes and it costed 5 bucks.
30, whereas Opus here took 13 minutes, so a little bit longer, but it was cheaper at $4.22. You can also see the breakdown here of input and output tokens and like what models they use and stuff like that. But, let me switch over to Codeex here. This is the actual result. So head-to-head, they both pretty much passed everything, which is great.
But Codex thinks that Fable wins here because Fable's production patch is exactly the oneline upstream fix blah blah blah. You guys can read through this if you want, but the final ranking here was that Fable and Opus did similar, right? But Fables was a little bit cleaner and immediately reviewable. But now let's take a look at the second one. So I did a very similar example on the second one where I gave them both, you know, the same prompt, the same codebase.
As you can see, here was the repo. Here was the bug. Here's example, expected behavior, blah blah blah. So, in this case, Opus took 20 minutesish and it costed us $6.50, whereas Fable on the exact same prompt, exact same codebase, took 12 minutes and costed us $8.73. So, let's go see what Codeex said about these results.
So, here, right, because Fable versus Opus, we actually had Opus perform better. Four out of four passed right here, whereas Fable only passed two out of four and left this unresolved. So the technical score for Opus was 93 out of 95 and Fable scored 66 out of 95 which is really really interesting and think about the fact that once again opus in this case was cheaper for us. The bottom line was both agents demonstrated strong repository navigation and independently found the core architectural issue but Fable's patch is functionally close and fixes the user-facing update bug while Opus delivered the more accurate thoroughly tested benchmark passing implementation. And one thing that I was really excited to see in this release blog from Enthropic, if I can keep scrolling down here and find the right spot, they basically talked about how there was a huge improvement in Opus 5.
Okay, let me just find this real quick. Opus 5 is much stronger at verifying its work and iterating carefully until it succeeds, which is huge. Like I kind of alluded to earlier, verification has become one of the most important things that you can do for your AI agents. Essentially saying, hey, don't stop until you hit this condition, and this is the stopping condition. Here's how you can test if it's actually done or not.
which basically means if you want them to not stop until you know a certain metric is hit like explicitly 10 out of 10 of this objective criteria or if it's something a little bit more subjective you can have them spin up sub agents that have to argue and debate and you have to keep going until all five of them come to a consensus or something like that basically just the ability for the AI to build something design tests to see if it's done or if it's good and then keep iterating until the test passes. So that's something I've realized as I've been testing out different models and different harnesses. It's like yes that does matter but at the end of the day what matters way more is how you instruct it and how you feed context in. So just keep that kind of stuff in mind yes it's good to find the best model but find the best model for your use case and then understand how to talk to the models right by the way guys as I'm editing this video I just wanted to say that at the end I go over like a snapshot of all of these experiments and see like total cost total tokens total time to run. So if you kind of want to skim through the experiments, feel free.
If you want to jump to the end and see like the total consensus, then that's there. So just want to let you guys know. But let's get back to the video. Okay, let's take a look at our next example here. So in this one, what I did was I said, "Hey / goal, build me a 10-second hyperedited, engaging, viral worthy announcement video for AIS Live." That was our live event that we just did.
And I told it that it could look through whatever it wanted in my codebase in my entire computer to figure out, you know, the information to use. So, let's take a look at the two examples and how much they costed us. So, this one was Opus, right? And I'm going to open up this example right here. This one actually created us a vertical version and a landscape version.
So, let's take a look. Okay, that was the vertical. Let me play the landscape real quick. Okay, so not bad, but not great. It only had 10 seconds to work with, and it was pretty fast-paced, and it honestly looked just a little bit like computery, like not super professional.
But anyways, let's see how much that actually costed. That took 40 minutes, and it costed $11.19. Let's go over to Fable and see what it did. Okay, very cool. So, that one, you know, they they had different sound effects.
They had different music. Also, what you'll notice here is that some of that was inaccurate. I don't know if you guys went to AI Live or not, but some of that data was outdated. Now, that's not Fable's fault. That's probably more on me for not keeping that context completely updated inside of my OS.
But, if it would have been better about verification, I'm pretty confident it could have found the right stuff. If it would have dug deeper into my school community and looked for threads, if it would have looked at some other LinkedIn posts or other things that I've done, it could have found out that some of that stuff was inaccurate, like some of those speakers didn't end up speaking at this event and stuff like that. So, there was a little bit of like a context issue there. But, as far as the actual videos, that's what we got, right? And they're they're different.
You can tell that the models have different taste. They were given the exact same prompt. Opus decided to make two. Fable decided to make only one. So, let's take a look at the cost of Fable.
This one was obviously, you know, about half the cost, about $7. and 7 minutes and 49 seconds. Um, so Opus took a lot longer here, but it did decide to do basically double the output. And by the way, at the end, I'm going to show a full breakdown of all of the total costs, time, input, output tokens, so just stay tuned for that. But let's keep flying through some of these other experiments here.
And by the way, if you want to access this entire free breakdown document, as well as all the other resources that I ever give away on YouTube for free, just go to my free school community. The link for that's down in the description. You'll go to classroom, you'll go to all YouTube resources, and then you'll find everything in there for free. So, let's get back to the video. Okay, so I think you guys get the point.
All of these other experiments, we're going to do the same thing. Exact same prompt. I'll show the cost and we'll look at the outputs. So, this time I asked for a onepage landing page for a certified AI consultant program. I told it it could look through whatever it wanted and, you know, make it as impressive as possible.
So, let me real quick pull up both of these outputs. Okay, so here is the Opus output. Not another AI course, the credential that SMB market hires from. You can see there's a bit of a dynamic 3D element in the background. It pulled my logo.
We have an apply button. We can jump down to certain sections. That's pretty cool. If I go to the nine pillars, you can see what this course is actually designed around. I mean, this does look pretty generic.
It has our brand guidelines, though. It uses our colors. It uses our logos. It uses our different buttons and our different design kind of criteria. And it has all of this, which is so far what I can see.
All of this is completely accurate information based on my meeting recordings, based on my, you know, internal docs about this cohort. As you can see right here, we've got this information as well. And yeah, um, this isn't too bad. It's very wordy, but it's also very accurate. So, let's switch over now to Fable's version of this and see what it did here.
If I can find where it kept this HTML. Here it is. I'm going to have to open this up in the browser real quick. Okay, so here is Fable's version. It's very similar.
We have the card right here, which is a nice touch because we do actually have like a card designed very similar to this. Not another AI course. the credential the SMB market hires from. Um, we've got nine pillars, two layers each, 54 job post analyze, 50 founding seats, claim a founding seat. We can keep scrolling down.
So, they're very similar style, right? This one has a nice little animation here. They both obviously pulled my logo and our brand guidelines that you can see that they're designed very similarly, which is great, you know. Um, we have other stuff here like the disciplines, the pillars are the same. Once again, I don't know.
I mean, they're they're obviously designed very similarly. This is a nice touch here. I think the thing about this is I probably would obviously want to manually tweak both versions. They're very similar. I don't think one definitively beats the other.
So, let's look at the cost and the time. So, Fable was 20 bucks.50 and it took 22 minutes whereas Opus was 35 bucks.83 and took almost an hour. So this is one of those cases where Fable was actually cheaper and quicker and arguably maybe just like a little bit better, but it was very similar on that side and very subjective. So maybe one conclusion we can start to draw here is that Fable still kind of wins on the creativity and the design side compared to Opus 5. Whereas right now Opus 5 is kind of having more of an edge for me on like actually following directions verification.
In most cases, it's going to be cheaper once again. Okay, let's keep on moving here. So experiment number three here was that I wanted a LinkedIn post with a LinkedIn carousel and you can read the rest of this post here but basically I was trying to raise the stakes right I was trying to say hey you know if I post this out on my audience um it would be bad if this was you know clearly AI generated or whatever and I also had it choose the topic so opus let's see what it decided to do here is the actual PDF it created for me as the carousel I don't love this styling right it's obviously pretty consistent with our brand guidelines but it just looks a little bit you meh. It just doesn't look super super professional. So that's the carousel.
It basically chose to write about AI got dramatically better at coding and trust in it went down. So let's see what the actual post looks like. It probably used my LinkedIn writing skill to do this. So here is the actual post. We've got some real stats in here.
We've got the arrows a reference to the actual carousel down there. Okay. And if I go to Fable version now and scroll up here. Aha. So Fable used a different style of carousel which I actually like a lot more.
It's kind of like that tweet style. AI agents went mainstream. Trust didn't. So, that's pretty interesting. It did similar research and they honestly both came to a similar conclusion on, hey, you know, based on Nate's audience and based on what's going on in the space, what should we write about?
Which is pretty interesting. But ultimately, I like this deliverable much better. And I mean, honestly, I think that when I read through these, the actual content of LinkedIn posts are pretty similar as far as like which one do I trust more. You know, I've got a skill built around it. I've got a no AI slop sort of skill as well.
And this is a perfect example of like writing a LinkedIn post, generating that content. I think even Opus 5 is overkill. Like you could write really good content with Sonnet, you know, Sonnet 4.5. So those are pretty similar. Let's look at the cost and the time.
Fable costed us $6.17 and took about 7.5 minutes. And Opus here costed us $8.22 and took longer. So, another example where Fable actually came in cheaper and faster, which is quite shocking to me because what that tells us is that Opus is using so many more tokens to actually be more expensive because if Fable and Opus used the exact same number of tokens, both input and output, then Opus would be pretty much exactly half the cost. But that's not the case. So, when Opus comes in costing more than Fable, it means that it was way less token efficient as well, which is a little bit concerning.
Okay, so let's just keep on moving here though because obviously all of these experiments I could run is not going to be the exact same as when you use it and you know at the end of the day it's a black box. You're pulling a lever on a slot machine. So let's move on to fable or sorry the the fourth experiment. So here I told it to go to my YouTube channel, pull comments and then to go to my school communities and look through threads and I want to understand what my audience is saying, what the pain points are, the number one product that I could build to help solve their pain points and the number one best YouTube video that would resonate with them. So, let's open up the HTML here for Opus.
What your audience is actually telling you. It pulled a bunch of sources. It pulled things from, like I said, right here. If I can keep scrolling up, 2200 comments on YouTube, 480 school posts, it looks like. And here's what we've got.
So, we've got an HTML. Once again, I don't love this font in general. I mean, this is on our brand guidelines, but I might want to change that because it just looks very typewritery. It looks very cheap, honestly. Anyways, we've got pain points that are ranked, pricing and scoping, proving the automation worked, ended in verse cloud versus co-work token, blah blah blah.
The number one product to build would be the offer engine. So, a cloud code skill pack plus templates that takes a discovery call and then produces a scoped priced offer with a working measurement layer out the other. So, a bunch of different skills. It tells us why. And then the number one video to make would be I sold an AI system to a real business in 7 days.
Real client, real invoice. Okay, so let's see if Fable came to a similar type of conclusion with what I could build. So once again, this thing looked through um YouTube comments as well as it looked through school posts. It looked through less school post though, which is interesting. Let me pull up this HTML.
So we have cost and token pricing, error stuck mid build, getting money or sorry, getting clients making money. It goes over the YouTube mood, the school mood, pain points once again, which don't seem to be the exact same. They're similar, but you know, they're not the exact same. The number one product to build would be the AI consultant kit. So, very similar.
A package client delivery system that takes a member from I can build automations to a business paid me, not another how to build course, blah blah blah. Okay, so that's pretty similar. And then I worked as an AI consultant for a real business, real client, real numbers. Okay, so these are very, very similar results. So, this would be a matter of which one do you trust more and maybe which one looked through more data and that's how you could maybe trust it more.
So, let's look at the cost. Fable here spent $10.60 60s and took 10 minutes, almost 11 minutes, whereas Opus spent $8.34 and took 20 minutes. So, a little bit less efficient once again from Opus, but ultimately ended up being cheaper. Okay, let's move on to the fifth experiment. I'm sorry if I'm going fast, but I also don't want to bore you guys just like really really diving into all these because there's a lot of things to go through, but I want to show you kind of a wide range of stuff.
So, this fifth one, your job is to create me a YouTube video outline and a slideshow, an Excal style presentation for this YouTube video. I want you to go through past LinkedIn post, school post, YouTube videos, my AIS plus Q&A. So there's a lot of things to dig through. And then I basically told it you are a product manager. You're in charge of agents.
You don't do anything. You just delegate work and you review stuff. So that's what I wanted. Okay. So it created the presentation and the outline.
Let's first look at the outline. So context engineering for agents. We have a cold open. We then move into what changed, why the terms exist, the failure modes of context, and we get into writing, selecting, compressing, isolating. Okay, so a pretty legit outline as you can see here.
Let's open up the actual Excal slide deck it made for us. Okay, so context engineering for AI agents. Your AI agent isn't dumb, your context is. Bigger windows didn't help, so harder to see. I like that little touch.
We've got our Scala style boxes here. Um, Andre Carpathy Anthropic. We've got a quote right here. Attention is a budget. And honestly, this doesn't look very branded the way my other Excal uh presentations look.
So, I'm not sure exactly what happened here, but this doesn't feel exactly right. Poisoning, distraction, confusion, clash, and rot. We've got some other stats here. So, not too bad, right? I would obviously make some tweaks before I would get ready to start, you know, thinking about how I'm going to present this, but not too bad, especially for one pass.
Okay, let's go ahead and see what Fable did here, what kind of topic it shows for us. So, if I go to it, it created outline, slides, and notes. So, I'm going to go to the outline context engineering for AI agents. Wow, very similar. Okay, so we've got the hook, we've got the section by section outline, what bad context is costing you, the four moves, right, select, compress, isolate.
Okay, so these are finding similar things, which is pretty interesting. I mean, it's looking through assuming similar data sources. So, that's kind of good to know, right? Like the consistency makes me feel good. This looks more like what my YouTube video ones typically do look like, though.
So, that means maybe Fable did a better job navigating into my other project and finding the right skills because I forgot to mention this directory was a completely fresh one. Both of these are working in completely fresh environments. So, it's not inside of my Herk 2 as all of my normal things are running. This one had 29 slides, so quite a bit. We've got um a big story here, which is something that happened to us.
We have these different colors here. This one looks way more like what I typically am trying to build. We've got this nice visual with context rot gets lost in the middle. You know, we've got these nice visuals here. I would say this one is definitely a better presentation.
So, once again, Fable is kind of coming in on top when it comes to like the actual visual elements. I'm assuming they both did verification loops of screenshotting and, you know, validating. I like these a lot. These are nice slides. Yeah, I like these slides better.
So, definitely I think Fable takes the cake here. Let's look at cost. So, Fable took about 8 minutes and costed us 40 bucks. Wow, almost 41 bucks. Forgot Opus.
Let's see. An hour and 15 minutes and 33 bucks. So, I don't know. I think that Fable wins here even though it was a little bit more expensive because it was still more token efficient and it was faster. Okay, let's go to the next one.
Number six. So, this one's interesting. This one's very interesting. I wanted to try to show you guys some computer use stuff. I compared it a little bit with Codex computer use.
And ultimately, I still like Codex computer use. I don't know why. Um, they're very similar now, but I think because I just have this bias, you know, already for Codeex computer use. I don't know. Anyways, I don't use computers a ton, but I do use it a lot for verification stuff.
So, here I told it to use Play CLI and I told it, this is something interesting, right? I told it to go to Google and play the snake game. So, I don't know if you guys you obviously know like what the snake game is, but if I come here, you can just play snake right here, which I used to do in class all the time. So, I told these two models to go to Google and to do that. I wanted it to only play five games and screenshot the score of each game and give us the average, right?
Okay. You know what's really weird? I actually just noticed something. So, I made a big whoopsies here. I I sent this off as Fable, but I only but I actually used Opus.
And then for the Opus run, where I thought I was using Opus, I was using Opus. So, I did Opus twice here. But I'm not even going to change that because I want to show you guys this. Same exact prompt to Opus, right? Same exact prompt to the same exact model twice.
But we got drastically different results in this first version, the one that I thought was fable. An hour and 53 minutes, 18 bucks. But look what I had to do here. You were told to only run five games and give me the average score there. I'm not sure why it went rogue.
This thing started running like 30 different games and it just kept running and kept running games. It was trying to like maximize itself right here. It got an average of 63. So, it was running batches of five games at a time and did that so many times. I was like, why why are you not following my instructions?
But opus is so much better. But I didn't realize that they were both opus. So, that's just a really good reminder that like at the end of the day, these things are completely nondeterministic. don't know what they're going to do. So anyways, this Opus run 2 hours $18 and then the real Opus run that I thought was Opus from the beginning.
Um 1 hour 10 minutes and $10. But this Opus run did much better which is really weird. 81 25 33 2144 which is like so so much better than the other one did. So I'm not sure exactly what happened there but anyways hopefully that was kind of interesting to you guys. Okay, so let's take a look at this last one that I have to show off to you guys today.
So, this one was um a bit longer, right? You can read this if you want, but basically what I wanted was a simulator where we could see different like buildings and vehicles and we could stress test them with different weather and we could add weights and we could even build and design our own um structures and then test them. And this one's pretty interesting, right? Because I gave this same prompt obviously to Fable and to Opus. But let's take a look at the two differences first and then how long they ran.
So, I'm not going to tell you which one's which yet. Let's just take a look. So, here's the first one. As you can see, there's a lot going on. Like, it's a little bit overwhelming, right?
So, if they wanted to build something that was kind of not intimidating, they failed on that front, but that's not exactly what we asked for. We can see the different nodes. We can see the different beams. We can add like weights to them. We can see if they're passing or failing.
I don't even know like this is one of my first times opening, you know, this. I just opened them both to look at them, but I didn't use them. We can see I can add like snow, right? So, if I come here and if we want to add snow, we can see how things are changing. And if I add some rain, um, this is adding it to the whole thing.
Gust factor, gravity, you can see it starts to change colors right there because there was way too much weight on here. And as you move this stuff, you know, I'm not an engineer, so I'm not going to come in here and tell you this is completely structurally accurate, and you could use this for um, you know, making sure that your stuff isn't going to fall and hurt people. This is a good simulation, right? Because you can see where things are being put under pressure and where you need to increase um some stability and stuff like that. Now, I will say this is pretty um overwhelming.
Like this UI, I don't really understand what to do, but if someone did understand how to get in here and how to test out this different stuff and, you know, build their own custom things, I could see this being very useful. And it just goes to show that in less than two hours, I already have this PC that I could go get feedback on and iterate on and stuff like that. We've even got these skyscrapers in here that we can start to add a bunch of different, you know, things to. So, anyways, this was the first version. We've got a bunch of different machines and then you could also build your own.
So, let's take a look at the other version. This one's a bit more user friendly, right? And this one honestly does look a little bit more AI made. This looks very like legacy software. This one clearly looks a little bit more AI, but you know, the UI I wasn't too concerned with.
But, this one's a lot simpler. I can see the different things. I can easily add wind or snow or an earthquake and I can see how much pressure is being put on these different points. You can see I can change to a skyscraper or a school or a vehicle. And we can do the same thing once again.
And this one also makes it way simpler for me. If I wanted to build and design my own thing. So if I wanted to add a few nodes here, I could add one there. I could have one there, one there, one there, one there. And then I can start to connect these, right?
So I can connect these here and here and here and there and here and there. and you know put some triangles in here if we really want to you know start getting fancy and building some nice support. So anyways this one just makes a bit more sense and I didn't prompt it to say hey you know like this should be easy to use and people should understand it and the UI should not be intimidating or whatever. Um but anyways like I can add weight to these different things and I can start to really put some pressure on this stuff and you know see what it's going to do. This is saying that this one is unstable.
So anyways which one do you think was which? because this one was actually Fable and this one was actually Opus, which I wouldn't have expected honestly because I think this one is honestly much more welldesigned when you think about what you're actually seeing in the data. So, let's take a look at how much these costed us. So, Fable, this only took Fable 7 minutes and it costed us 73 bucks. So, it was just that that just goes to show how quick Fable can really run your session if you're not being careful.
And then when we go to Opus here, this one took us 2 hours and 26 minutes and 112 bucks. So clearly Opus got stuck in a different loop or had for some reason opus interpreted the verification criteria a little bit differently and it ran way more sub aents and it did more stress testing on it. And maybe that just goes to show this line that we talked about that Claude Opus is much stronger at verifying its own work and iterating carefully until it succeeds. So maybe that just goes to show that in action right there. Because the thing is, I think that Claude Fable could have easily designed something and built something this level of detail and much better.
Like far far better. I've seen Fable do things that are way more incredible than this. And it could have, but it just decided for some reason based on the way I prompted it or whatever it was, it just decided to be done. It decided to be done here, which as you guys know, if you've played with Fable, it's capable of so much more. But I think that that's a really good reminder of the fact that once again, these models are nondeterministic.
but also that Opus 5 interprets things different than Opus 4.8 and interprets things different than Fable 5. Which means the first thing that I did when I got Opus 5 is I ran my skills. I ran my regular workflows. I ran my regular things that I do, you know, generating some YouTube stuff and helping me out because I wanted to see how it feels. That's why I always say I take these benchmarks with a grain of salt because it matters way more about how you talk to it and how it feels and how you prompt for verification and all of that kind of stuff.
Okay, so here are the consolidated results from those sessions that I just showed you guys. Keep in mind, I accidentally ran 10 with Opus and eight with Fable. It should have been 99, but either way, the numbers are still pretty telling. Look at this. Opus 5 spent more, which once again means that it's more inefficient or I could have just said less efficient with spending tokens because Opus 5 per token is half the cost of Fable 5.
So, for it to be more expensive means that it's using way more tokens. We can see the combined output tokens right here. 2 million for Opus and 832,000 for Fable. We can see the active time for Opus was 630 minutes. So an average of an hour for all of these sessions.
Whereas an average of about 25 minutes for the Fable sessions. Here's the total. Now we can see on the output speed, we have some stats here with Opus and with Fable, the cost per minutes with Opus and Fable, the cost per thousand output tokens, Opus and Fable, API calls per session, and tool calls per session. And just a breakdown of where the money went. So it's a pretty similar split, right?
Cash reads a lot of it and then we have cash right and then we have output and input. And on both of these and on all these runs, the input was pretty minimal because it was basically just, you know, um creating output. And if you look at all the sessions from most expensive to least expensive, it's honestly pretty split besides the fact that Opus here owns like the most expensive one. But anyways, I will attach this exact document in my free school community if you guys really want to check out this, you know, this data session by session. But what I really wanted to see was this headline up here.
This headline is super interesting to me when it comes to the output tokens and when it comes to the time for actually running these models. So once again guys, I hope that you enjoyed this comparison here where I showed you some actual things that I have done. Like I said, I re really recommend that you just get in here with Opus 5 and you run your skills and just see what feels good and what doesn't. have it do some verification loops and see if you like the way that Opus feels as an orchestrator. I still personally like the feeling of having Fable being an orchestrator.
And I like to say something like you shouldn't be writing any code or executing anything. You should just be telling Opus what to do. And that saves your session limit with Fable big time because you can be running Fable for multiple hours or even a day and it won't even hit like 300K or 400K on the context because all it's doing is delegating. So try out that tip. But also, you know, start using Opus for that because I think a lot of us don't realize how powerful like an older sonet model really is that you could use that to drive most of your daily knowledge work.
And sometimes Opus 5 and Fable 5 are both overkill. So really think about what you're doing and matching the intelligence of the model with the intelligence needed for the task. But anyways, I hope you guys enjoyed this video. I hope you found it insightful. And if you did, please give it a like.
It helps me out a ton. And as always, I appreciate you guys making it to the end of the video. And I'll see you on the next one. Thanks everyone.