GLM 5.3 Flash vs GLM 5.3: When Cheaper Is the Right Call

0:00 Okay, in this video I want to talk about the latest GLM 5.3 Flash, but rather than just show you some tests of it and things like that, I actually want to talk about how this came about and how

0:11 it actually compares to their flagship GLM 5.3 model. Cuz really, we've got two stories here. One is about how to squeeze the most amount of performance out of your old base model with RL. And

0:24 then the second one is basically how do you build something completely new with a new pre-trained base that's going to be built around efficiency. I'm also going to talk about when you should use

0:35 the flash model versus when you should use the full-size model and also where this could all be going not only for ZAI but for a lot of their competitors as well. So I made a video about GLM 5.2 2

0:47 and that model seriously impressed me and I was actually in China when they launched 5.3 which is also a really impressive model. So the key thing with the 5.3 model over 5.2 was that ZAI

1:01 really focused in and dialed in improving their RL for particular tasks specifically for stronger coding for agentic tasks and also for cyber security users. Don't forget it was GLM

1:17 5.2 was the model that HuggingFace had to use to defend against the OpenAI agents that were attacking them. Basically, because none of the other proprietary models would actually

1:29 entertain the tasks that the HuggingFace team were trying to get the models to do to defend their own system. So, in some ways, you could say GLM 5.2 saved the day there. And it does seem that Zai

1:41 took away that they should even improve that. If we look at the 5.3 benchmarks, we see clear advances on things that they were focusing on basically improving their pipeline. Now, if we

1:52 jump to 5.3 flash, we've got a whole different model here. This is not the same base model. It's not even a pure direct distillation to get a smaller version of that 5.3 model. For a start,

2:05 the size of the models is quite a lot different. So, you've got GLM 5.3. That's a mixture of experts. In fact, both of them are mixture of experts. But 5.3 is 744 billion parameters with 40

2:19 billion parameters active. And that 40 billion parameters active means that this is still quite a chunky model to run. Jump forward to 5.3 flash. They've now got a model down to 320 billion

2:34 parameters with only 18 billion parameters active. So this is effectively sort of the halfs size model of 5.3 but ironically it's actually pre-trained on more data. Now one of the

2:47 biggest differences between these two models is that 5.3 is a texton model and 5.3 flash can handle both text plus images and videos. Now when we look at the pre-training 5.3 flash actually was

3:01 pre-trained on more tokens than 5.3. They're saying that that was over 30 trillion tokens there. Now it is important to emphasize that those 30 trillion tokens are multimodal. So when

3:12 compared to the 5.2 base that was used to make the 5.3 model that was only 28.5 trillion tokens but they were all text. So that size difference translates very much into the cost of using this thing

3:28 as an API. Now I'm not going to talk about running this model locally in this video. I'm going to actually save that for another video and do a whole video on how you can run this locally and what

3:38 kind of speeds you will actually get for the 5.3 flash. By using this model as an API, you're looking at something that's almost 1 nth the cost for the output tokens. So the 5.3 flash full price is

3:52 50 cents per million tokens out and 15 cents per million tokens in. Now, they're currently running some promos on Open Router to make it half that price. But let's focus on the full price

4:04 because at 50 cents per million tokens out, that saving is huge compared to the $440 per million tokens out for the full-size model. Now, while the size and how they were made are hugely different,

4:18 these models also share a lot in common. Clearly, there's a lot of similar data that has been used in the pre-training of these and they also share the same API behavior. So, the 5T flash supports

4:29 controlling your thinking token budget through the reasoning effort parameter which means you've basically got low, high, and max and that's identical to the full size 5.3. And unfortunately, on

4:42 both of these models, you don't have the ability to turn off thinking. So, that one's definitely a change from where GLM 5.2 was. Another thing that they have in common here is that despite them having

4:53 different base models, they clearly are sharing a lot of the same postraining, probably using the same RL environment stack for both the 5.3 and the 5.3 Flash. Now, another interesting thing

5:06 about the 5.3 Flash is often companies use these smaller models to test out an architecture change at just a smaller size to see if it's going to work before they actually train the full one. And

5:18 that could be precisely what's going on here. GLM 5.3 Flash is really designed around this efficiency kind of architecture. They're using a hybrid combining both sparse attention plus

5:31 linear attention here and is interle those two types of attention through the 45 layers of the model. And that's actually a little bit more than half the 80 layers that the full 5.3 is using. On

5:43 top of this, they're routing through eight experts of the 288 experts that they have each time. And it's great to see that they've got a multi-token predictor draft layer in there as well.

5:54 And if we come in here and look at the benchmarks, we can see that this model is outperforming GLM 5.2, which is kind of insane considering that this model is half the size. You can choose to run it

6:06 locally with a lot less hardware than you needed for GLM 5.2, too, but mostly whether you just use this locally or whether you use it as an API, you're just getting better performance out of

6:16 it overall. Now, one of the things that kind of annoyed me was that they don't really show a lot of benchmarks where they're benchmarking the 5.3 Flash versus the 5.3. Now, when we look at the

6:27 benchmarks that they've posted here versus sort of comparative models, we can see that this model is giving models like Opus 4.8, 5.6 6 Terra, remember that's the mid-level GPT nowadays, and

6:41 Gemini Flash, it's giving them a run for their money. Not always beating them, but for some of them, it clearly is beating them. On top of this, the ones that we have where we actually do have

6:52 comparisons against GLM 5.3. We can see, for example, that it's actually beating the GDP val for 5.3. But clearly, for some things, size really does matter. So we see for example on the humanities

7:06 last exam the 5.3 flash is getting 55.3 the full version of 5.3 is getting 62.5 and rightly the 5.3 they're comparing against Fable 5 and GPT 5.6 Soul in there. The simple fact though is for 1

7:22 nth the price here you're getting a model that is insanely good and has vision abilities compared to the full 5.3. And honestly, I kind of feel if I was using this with agents and this

7:35 model was going to fail for the agents, I'm probably not going to fall back to the 5.3. I'm going to fall back to one of the frontier models from OpenAI or Anthropic. Okay. So, when we come in and

7:45 look at the artificial intelligence scores, we can see that this is scoring super high for one, a model that's open weights, and two, a model that's this size. Don't forget, sure, it's been

7:57 beaten by the Quen 3.8 8 model, but that model and the Kimmy K3 model, etc., those are well over two trillion parameters in here. So, Z AI is doing kind of amazing for the size. They've

8:10 got the GLM 5.3 model at an intelligence score of 60 and the 5.3 flash model at an intelligence score of 57. And if we compare it to some of the other models that you can run locally, really the

8:22 ones that are most interesting are probably the Quen 3.8 8 flash next, which I'll talk about in another video, and the Deep Seek 4 Flash. So, when we look at this, really, it looks like this

8:33 new 5.3 Flash and the Quen 3.8 Flash are in many ways a whole new class of models that is small enough where a lot of people are going to be able to run them locally, obviously with some pretty

8:44 beefy hardware, but also small enough that any endpoints are going to be very cheap for this. If we look at the agentic index in here, it's looking even better with this really just behind a

8:55 few models, mostly the anthropic models etc. But also well ahead of Kimmy K3 which is that 2.7 trillion parameter model. Now where the performance actually slips a bit though is when we

9:07 compare the intelligence index versus the output tokens here. So really we would like to see this being able to get this level of intelligence without a huge amount of output tokens. And you

9:19 can see that both for this model and the Quinn 3.8 flash both of them are using a lot of tokens to get that level of intelligence. Okay. So first up the thing that I was most interested in

9:31 testing were the things like function calling and just seeing how well it does on the various agentic tasks. And so when I tried this with max reasoning, it does fantastic, right? You can see it

9:43 passes all the tasks really well. So then I thought, okay, well, what about if we try it on low reasoning? And on low reasoning, it's obviously it still can do the tasks, but for each of the

9:55 tasks, it does exactly what I've asked it to do, and it is able to basically get the job done without a lot of thinking. Now, it's interesting that they've trained this on sort of a clawed

10:06 code harness, and it would be really interesting to see how well it does with other harnesses that are out there. But even here with the low thinking on longunning sort of jobs and stuff like

10:18 that, it does very well. And it understands when prompted to not use tools and just use the model itself, it's able to resist the urge to use tools there. Next up is the Aentic Pro

10:29 test in here. So this is testing a lot more sort of long horizon sort of stuff. This time I started with the low thinking on. And it's amazing that the lowing on is kind of often close to no

10:42 thinking on that. It will literally just have one line of thinking and usually even less than 50 tokens of thinking. That said, all of these tests it was able to do really well. One of the ones

10:54 that I was interested in the most is this failure retry. So this deliberately gives failures for the first couple of times that it does the thinking and to see does it actually give up right does

11:07 it keep persisting and going on right and you can see the thinking to itself was call tool if fails retry unbelievably small amount of thinking tokens but it did do exactly that and it

11:20 was actually able to deal with multiple errors coming here before be able to execute in there then testing it with different distractors for sort of toolb it doesn't fall into the challenges that

11:32 a lot of the models do here. So overall the agentic stuff is really good. So if we come in and look at the SVG tests, you can see that even on low this is doing a really nice sort of Pelican one

11:46 on the max setting. It does a fantastic job here, right? I got to say this is definitely one of the nicest ones. It's got massive amount of thinking though going on here, right? just really huge

11:56 amounts of tokens. And then the whole thing about it overfitting is pretty clear. You can see if I do a red dragon with a very similar prompt. Not the best looking red dragon here on low and then

12:10 on high. Perhaps not a great red dragon. Bit better and stuff like that. It's got tails. It's got the wings. We've got it flying at night. But you can see that this took a huge amount of tokens to

12:22 actually generate out. And we can see that like it actually does have some nice chain of thought in there. Definitely going through the chains of thought for this. I'm noticing different

12:33 patterns compared to where Quen is going. Quen is going much more into the abbreviated thinking ideas. Sort of the caveman thinking in a similar direction that GPT has gone. This one doesn't seem

12:45 to be at that level yet. And that may be one of the reasons why it uses so many tokens for a lot of the tasks. Testing it out with low mode on things like rag, it does a really nice job here, right?

12:57 That it really is not using a huge amount of tokens for this output at all. It's very much to the point for those particular low tasks, etc. And then finally, looking at the design stuff, I

13:09 find that for the low tokens, it is very sort of rudimentary designs. Not that they're really bad, but there's just nothing visually appealing in a lot of this. Now, that could be the prompt. If

13:22 I gave it a much better prompt or something like that, it's probably going to do better in that sense. Giving it more tokens, whether it's sort of X high or something. It definitely goes on for

13:31 a lot more thinking and we get nicer quality kind of stuff out. Interestingly, the design and the color theme that it comes up with are very much things that we've sort of seen

13:41 before. And also nice to see that it's using some SVGs and stuff in there as it sort of makes some custom graphics rather than just use images. But we've got some weird things like alignment and

13:52 stuff like that going on here, which seems a bit crazy to me. And things like when we change the quotes, we're actually dynamically changing the box rather than keeping it the box the same

14:02 size throughout. In regards to the pros test, it seems to be quite decent for that kind of thing. I'm really thinking now that all the models need to be sort of guided a lot more by certain skills

14:13 that you use for those kind of tasks to actually be able to elicit out the kind of writing that you want for this. Now I also tried it on running some MCPs and stuff like that and it does seem very

14:25 good at using that and running it in some basic code harnesses like open code etc. It also seems to be doing a good job there. The big thing I wonder with this though is if you want something

14:35 totally private, do you go for this or do you stick to the Quen 27B model which still is an amazing model and last weekend alone I generated about 100 million tokens with that model locally

14:49 running 24/7 and I kind of feel with that if somebody actually does fix the long thinking there that's going to be a killer local model for a long time to come. Okay, so lastly, just before I

14:59 finish up, one of the things with this model is that because if you're using this in max mode, it will think for a very long time and overall that can just make it slow. So the issue there is

15:11 you've got a decision to make if this fails on a particular task and you've got a verification step or something that's actually checking and it's failing, you've really got a decision to

15:21 make of how many times do you want to try it again before you go to a better model. And the challenge there is that if you try it many times, there's a high chance that it will actually perform the

15:30 task that you wanted. And the issue with that is not that it's going to be expensive, it's just that it's going to be slow. So you really need to decide, do you have something that's going to

15:41 need a real-time response, which in that case, you're probably going to go for a different model, or are you prepared to write it out for a few times and try to get the model to fix itself before you

15:50 go to a more expensive model? Because remember, even if you're going to go to the GLM 5.3 model, you're talking about something that's around nine times more expensive for the tokens that it's going

16:02 to output. So overall, this is a really fantastic model. It'd be great to see this supported by someone like Cerebras where they could actually perhaps run it a lot quicker than other people have

16:12 been running it. And like I said earlier on, I will make a video of actually running this locally so you can sort of see what speeds do I get compared to the APIs out there at the moment that are

16:23 only getting around sort of 50 60 tokens max themselves. So let me in the comments if you've tried the model out. I'm curious to see are other people also finding that perhaps this is not as good

16:33 at design as some of the other models recently, but is very good at the agentic steps and the function calling and tool calling that go along with it. Anyway, as always, if you found the

16:44 video useful, please click like and subscribe, and I will talk to you in the next video.