Hy3 from Tencent - The NEW GLM Competitor

All right. So, yesterday 10 cent launched their fully complete version of the High3 model. So, this is a model that's been in preview for a month or two already. Now, when it came out, I wrote an article about it on Venturebe. If you didn't know already, I co-host the Venturebe podcast and also write for them occasionally on topics that I think are kind of important. And this model, I think, is important. And this is also why I wanted to make a video about it here. So, the Venturebe article is definitely directed at people making decisions in big companies about these models. In this video, I want to be a little bit more broad, but I want to go into why I think this model is important and also who should probably not even pay attention to this model. In many ways, this model is Tencent basically declaring that they also want to be a frontier lab and that they're also going to get behind the open ecosystem. So, Tencent has released a bunch of models in the past and they've definitely had a really strong deep learning team there. They tended to make things that they were using rather than perhaps focusing on building things for the open community like Quen has, like Zai has. If you don't know who Tencent are, they're a Chinese company. They're very, very big. Let me put it that way. So, probably the biggest thing that people know them for is WeChat. If you live outside of China, if you've never been to China, it can be really hard to grasp just how powerful an app WeChat is. In my take on it, it is the only super app out there. And as much as Facebook, Meta X, whoever in the west is trying to make a super app, none of them have even come close to what WeChat can do. This can basically have mini apps. It can do payments. It can do a whole bunch of different things. On top of that, Tencent has always been very big in gaming. A lot of people perhaps don't know that Tencent actually owns the company Supercell, which makes Clash of Clans and a lot of popular games out there. So their deep learning team has been very focused on building what it is that they need. Now with this release that may have just crashed into what the open source community wants to have as well. So let's look at what they actually released and then I'll tell you straight away who shouldn't be using this. And if that's you, you're most welcome to just turn off the video there. So they've released a 295 billion parameter mixture of experts model. It's got 21 billion active parameters and on top of that it's also got a 3.8 8 billion parameter speculative decoding model in there to speed the whole thing up. Now the context window goes out to 256K, a lot smaller than things like GLM 2.5, etc. And I think this brings us straight into who and who should not use this model. This model is very good at specific things related to agentic tasks. It's not necessarily the best model at coding. In fact, if we come in and look at their benchmarks here, they're quite honestly showing that while they're beating GLM 5.1, in the meantime, GLM 5.2 has come along and is beating them at most of the agentic coding tasks. So, if you just are looking for an open model that you can use for coding and you don't really care about anything else and you're using it through open router or something like that, then probably stick with GLM 5.2 over this. The benchmarks where they're winning for a lot of these though are things that relate to working agents. Now, even for those and for some of the reasoning benchmarks, they're not always winning against GLM 5.2. The main thing I think that sets them apart from GLM 5.2 is that they're well over half the size of that. So while it's still far from being a small model at 295 billion parameters, it's taking the fight to what I would say are the mid-tier models. So the models that are sort of 200 million parameters up. And if you look at their scores, their scores are very respectable compared to the deepseek models at the moment. So where I see this model becoming important is for companies and for offices where you want to have one model inside the office that's fully locked down that's fully local and perhaps even fine-tuned for your particular use cases. You are going to be able to do that with this and you're going to be able to do it on hardware that you can actually run locally. Now it's not going to be cheap but it's going to be in the realm of possibility compared to having to go out there and buy a few nodes of B200s etc. Now the cool thing is currently the model is free on open router. You can use a version of it free on open router. That's for the next two weeks. I would recommend that you do come in here and just try it out. It's not going to cost you anything. And then from that you can get a sense yourself of what this is actually good at versus what it's not good at. Now there is currently one paid version of this up there. It's probably too early to sort of know exactly what the pricing is going to pan out to be for this, but with it being well under half the size of GLM 5.2, too. I expect that it's going to be very competitive against GLM 5.2 pricing-wise. So, currently I have a number of agents that are using the Deepseek Flash model. I'm going to try those out on this. I think those could be quite good at this. And even just general agents that perhaps before you were using GLM 5.2 would make sense to try out with this model as well. The other thing that I found quite interesting in here is that Tencent themselves don't seem to be pushing the benchmarks, which makes me feel that this is probably not benchmaxed, right? It really does seem that they're going for a much more broadly useful model that's good at things like tool calls, output formats, stuff that you could imagine that they want to use themselves. And at the same time, they seem to have put quite a lot of effort into actually reducing hallucinations in this model. And you can see that here where they're talking about the, you know, comparing this to the preview model, they've managed to have both hallucinations and common sense error rates, which seems to have come just through better post-training, cleaning up their data, a lot of those sorts of things. Now, I don't know if we're going to get a full paper about this, describing what the actual training methodology was. Tencent has had a good track record in the past of releasing good open papers, but once upon a time, so did Google and Open AI, and that's just not the case anymore there. Unfortunately, at the time that I'm recording this, Artificial Analysis hasn't released any benchmarks themselves. So, we've just got what Tencent has actually released. And for me, these generally seem to confirm my beliefs about this that this model is leading in its sort of bandwidth of middlesize models. It's still going to fall behind some of the bigger parameter models like the GLM 5.2, like the Quen 3.7 Max, and the proprietary models that are out there. Anyway, let's jump in and have a play with the model and see what it can do. All right, so coming in here to test. You can see I'm using the free version here and you can really tell that it's free by the fact that on this particular run time to first token was 90 seconds. So I think that kind of shows if you're getting something for free, it might take a little while for you to actually get it. So the first test that I started out with was the SVG test, the Pelican test. You can see that this had 12,000 tokens of thinking in this. And that's a lot of stuff going on in there in the thinking, but when you look at it, it produces a very nice pelican on a bicycle. The bicycles sound. We've got some pedals. Kind of got a chain. We've got wheels. We've got everything going on there. And this certainly is a Pelican. Now, [snorts] what I thought I'd do is just show you running that same test on the preview version of the model. So I was curious to see how much had changed between the sort of old version which I don't think was fully trained yet and this one. But if you look at the output we can see that yeah clearly the output is not as good either. The Pelican is perhaps sitting backwards on it. So that was something that I noticed. Definitely it seems that the newer version of the model just has a lot more training going on there as evidenced not only by this particular test but by some of the other ones as well. All right. If we give it a the standard sort of 5,000word essay test, you can see that this time time to first token actually was back to being quite normal. I think it depends on how many people are using it at the particular time that you do it. It basically does a very nice outline for this particular task. And you could kind of think of the outline in many ways as almost like an agentic planning sort of step here which not a lot of the models do. In theory, this should help it to get a much better output here. So if we look at the actual output. So looking at the final result out, we can see that okay, we've got the introduction in here. We've got part two. Let me just take all of this. And we can see it actually just came in at a bit under 3 and a half thousand words here. But the thing I found interesting in this one was that it actually says due to constraints in generating a single continuous block of 5,000 words in one response, this text provides 2 and a half to 3,000 words. and then it really should go back to sort of flesh it out to 5,000. So again, sort of hinting that maybe some kind of a gentic revision step would be a good thing to look at here. All right, if we look at it at a simple sort of logic puzzle, we can see that it's turning out a lot of thinking. The thinking tokens here will actually be different. This is basically just using the JavaScript tick token rather than get the actual tokenizer for each model. And you can see the final result. We've got it coming back with some verification and we can see it looks like that it's got the right solution there and it's used a decent chunk of thinking to actually get to that solution. Now, as far as I know, we don't actually have any sort of control over the level of long chain of thought on this model. So, I haven't actually seen anything either where we can actually set the thinking to be off or on with this. But for me, it definitely has very long chain of thought for the thinking. That chain of thought seems to be very high quality, which is really nice to see. And I would say it's probably even better quality than a lot of the other open models out there. It doesn't seem to consistently have the same sort of let's verify step-by-step chains of thought that the proprietary models from Google, from OpenAI tend to have. I could imagine a high 3.1 actually doing a lot better just with them focusing on getting more bang for buck out of the actual long chain of thought that they've got there. All right, just quickly I will test it with an HTML coding test. Okay, so we're testing here for a polished home site for Dario's wellness retreat. If he no longer has access to Fable or his other models, could he use this model? And we can see that actually we get a pretty nice website out of this, right? It's certainly gone and gotten some good images. It's done the basics at least of putting this together. I'm going to need to adjust the UI, I think, to see more of the website at one time. Let's look at how the older preview version actually went. And we can see that it's got a it's done probably similarly well. It's got a different vibe though to it, which is kind of interesting. It's also been able to get images, no problems, although it seems like it may have missed one there. It's also got some nice quotes. Oh, and this one has actually got an opt-in form. Okay, I'm just going to quickly hide the phone number and the website that it's actually put there. But we can see that this has actually done quite a good job at fleshing it out as well. Okay, if we come in to test the uh Gentic sort of tool use here, I've got the standard 10 tests. The issue that I'm finding running these is not about the model. It's just about the free version often will time out on certain things and will be quite slow. So coming through these, it handles the tool call stuff, the multiple repeated tool calls very well. It does a really nice version of sort of formatting as well in its responses. Very nice, very good quality sort of formatting going through this. You can see here it passed seven out of seven. And the ones these other ones, it basically is just erroring out. And this is basically rate limiting because I'm on the free model. If I just run those three tests again, we can see things like the the no tool restraint passes very well. It's very concise in its answer and even in its thinking there. We can see the longunning pageionation with 12 tools also passes no problem at all. And the polling of products and stuff like that also passes no problem at all. If we look at rag, so this is basically where I'm assigning the high3 model to be the rag model and GLM 5.2 to be a judge for this. I put in an article. GLM 5.2 will write the questions based on the article and then Hi3 has to basically answer those and then I'm using the Ragus framework of thinking of how to basically grade those answers with GLM 5.2. Now [snorts] the main challenge I'm facing again here is not the actual model, it's the end point that we're getting rate limited. We're getting slow responses etc. When the model does respond, we get pretty good decent time to first token and pretty good token speed also coming back. And if we come and look at these answers, we're getting a nice sort of thinking of how to actually do the answer. That's what I'm always looking for. We're getting the raga scores below here. And we can see that the model is actually doing very well on these. The issue here is just that we're probably not the only ones taking advantage of the free tokens at the moment. All right. If we move from these, just one last test is is a sort of aentic pro test. We can test a bunch of different things in here. The ones I'm going to focus on here. We can also set a judge model here for sort of checking any outputs. The ones I'm really interested in here are the sort of tool failure tests. We're testing to see does the model keep going? Does it insist on what sort of we wanted and keep going to get this kind of response. A classic one here is where the testing framework is deliberately set to return these kinds of errors and then to see okay how does the model actually respond here? And we can see that okay after two of these errors it's not giving up right it's basically doing retry. Another one is does the model get distracted by things that are not relevant coming back in the API calls from the tools as well. So we can see here that we're asking for one specific thing about the answer. This is actually the response from the judge model up there. But we can see okay it's very quickly it sort of worked out what it should be. calls the invoice and then finds out and even though it gets other information back, it's not distracted by that, which is a good sign. Lastly, I wanted to try another test that I saw people posting online was this whole thing of getting it to simulate different kinds of little sort of HTML 5 games or things like that. So, I've got another one of those as well in here. Okay. And given a few minutes, you can see that it did produce the game. So, it it took its time to do it, but it certainly was actually making it happen in there with 29,000 tokens. So, just to finish up, look, this is a model that you probably should check out while it's free. You really don't have a lot to lose. Just try it out on your agent, see how it actually goes and how it performs, and then let me know in the comments what you're finding is good, what you're finding is not good. I do think the more of these models that we see that are gradually getting to this sort of mid-tier level of a few hundred billion parameters, the more that people start thinking about serving these things locally. Even if at the moment that hardware is expensive and cumbersome, you got to think it's going to get better over the next 12 months. All right, as always, if you found the video useful, please click like and subscribe and I will talk to you in the next video. Bye for now.