Ling 3.0 Flash First Test – A Surprisingly GOOD Coding Model!

Let me just make sure like that's not on me. That was totally on me. The speaker was off. So, oh, today we're going to be taking a look at an exciting new model because it is the perfect size for unified memory systems like the DGX Spark Stricks Halo or Mac Studio or Apple Silicon systems with 128 gigs of RAM. Now, this is Ling 3.0 Flash from Ant. And they have released a bunch of other models as well. They even have a 1 trillion parameter flagship model that you may be familiar with. So for today's video, we're going to be taking a look at this model and seeing how it performs on some tests that we're familiar with the outputs of so we can see what to expect from a model that is based on their own writing here soon to be openweight release on hugging phase. So I want to quickly point that out initially because if we scroll down a bit here some folks had mentioned okay is this going to be open source and they respond to that coming soon a twist dance might accelerate it. So, the weights are going to be coming soon and that is pretty exciting because this is really a perfect size for those unified memory systems. So, before we get into it, please do feel free to subscribe and also this may be a little shorter as a video than normal. I did just get back from San Francisco where I was in a march for open weights models with Clem from Hugging Face and a bunch of other open source enthusiasts as well. So, that was very fun. Without wasting further time, let's take a look at the benchmarks and then we'll just get to testing this where it is currently only available for free until I think August 3rd on Open Router. Now, in taking a look at the benchmarks here, I do have to say some choices were made being that this is also being compared to the Ring 2.61 trillion parameter model and being that they have the same exact logo makes it a bit difficult to assess which is specifically being referenced. But they do mention this is outperforming that one in a lot of scenarios. So, I'm going to just assume that the higher benchmarked version of this logo is the new flash model that we're going to be testing today. And really, without looking at all of these with a fine tooth comb, it does seem like this may hypothetically be trading blows with Miniax M2.7, which is a model that's like a little less than 100 billion parameters more than this one. So, if that is actually true, that could really be a fantastic option for unified memory systems when it is openweight release, of course. Additionally, they have some other follow-up posts right here talking about parameter size and things of the sort. Context is 256K, but it is extensible to 1 million. Additionally to that, they also talk about parameter size, which is 124 billion total with just 5.1 billion active per token. So, it should run fairly quickly on some more memory bandwidth constrained devices even. It is, as they say, available on open router till August 3rd for free. And then following that, hopefully the weights just get released and then we don't have to worry about like where to access it. But I suppose before we even really get into that, we should probably see how this performs. So all I've done is just set this up through open code from within the open router thing right there. Right here we have the browser OS test v2.5, which is the traditional one we run where it needs to make the GTA clone and things of the sort. This test is very likely made its way into a lot of training data by this point. However, it will still give us a reference point to see how this model compares with every other model we've tested in the last 6 or so months. So, we see it starts out right here just with a pretty dense to-do list about everything it's going to do. So, I'll be interested also in just seeing how this behaves throughout the process of actually creating the code, not just looking at the code itself as the end result. All right. So, I like seeing that it did generate the file very quickly, but again, if it's being hosted by someone else, especially for a model that will ideally be self-hosted, I don't care as much about speed because we want to see how it actually does on like something like this and not necessarily a cloud provider. However, I do like that it spit out like a 1900 line script, then said, "Let me review and fix issues in the file I just generated." So, it's looking for issues, identifying bugs, and then trying to fix them. So, it's proactively going through this before we even see the result, which is nice. So hopefully we don't have to copy paste back and forth between developer tools. All right. And it should just now deliver it for us. Good. And it did also hypothetically fix a bunch of issues that it would have noticed. Okay. Maybe not enough. But we have Nova OS. All right. Um sometimes a a missing background can make something appear worse than it is, but the clock showing no time is always a telltale sign that something more is broken here, which is the case. So all right. Unfortunately, we're not starting out on a very positive note because it just basically the entire thing was broken. So, I'm giving it a description of some of the things that are broken as well as the one pertinent error that's showing up here in the developer console. So, we'll see how well it can do in actually fixing this. Good. I didn't really see that it like completed the task there, but now we can see this is definitely an improved result. There is no right click, but that's okay. The clock is now properly showing the time in the local. And we also have okay virtual desktops. That's probably our special feature. So, we'll hold off on that. Let's start with start. Good. And we have search. This is what I hope to see from the get-go. So, good. Good. This actually for the size of this model, this is actually a pretty strong result, especially considering how few active parameters this has. Again, it's likely that this has made its way into a lot of training sets at this point because of all the copycats on YouTube now, but I'm kidding. But nonetheless, um, the copycats are on X, not on YouTube. It's still an impressive task because they still have to make it like look good. Oh, okay. I messed that up. Documents. I feel like this PC and documents may have been the same thing. Terminal, a very interesting showcase of all the potential commands there. Usually we have to type help to have that happen, but different, but we'll go with it. All right. Nova OS 1.0. Can we make a new directory? Oh, but it doesn't show it as a new directory. That's okay. Settings. Okay. We have some nice selectable gradients. I think the first one was probably the best. We can change the accent color. We can change. Okay. So, ideally, we'd see like a color hex picker show up when we do that, but that's okay. We have switch virtual desktops. That's our special feature. So, I'll hold off on that calculator. 51* 9 4 59. Yeah, B2. Why did it Oh, did it just like genuinely bug out at a multiplication problem that was too much for it? Eight times six. Okay. Eight. I just want to see like if there's a threshold here. Okay. No, it's just seemingly all multiplication. That's all right. It happens. GTA clone. Surprisingly decent for the size of the model. All right. That's a fast seven kilometers per hour. Can we pick these things up? And do they? They They do. Okay, so there's actually some interactivity here where we pick these up and they give us cash. I think that's a police car chasing us. How's the resize doing? That's not actually half bad, especially again for the size of the model. Next up, shooting range. Um Oh, wow. What the heck? Okay, we we hit one of these things. The problem is the cursor doesn't specifically lock to the window, so it's pretty easy to move it out of it. But I did see one of these things actually working. Maybe we're supposed to just still very different from the usual space shooter. So, that's good to see. And it's always fun testing models that are like different. They're not like fine tunes of existing ones or other things like that. All right, our final thing was this virtual desktop thing. Okay, and we just have like different ones. So, what if I hit two? All right, that doesn't seem to be 100% functional, but it it fixed the issues that were preventing this from even loading in the first try. So, that's very good to see. And some of the results here were competent enough. I'm now giving this the self-contained C++ skate game. This is the one with the California Boardwalk aesthetic. And I'm not starting it from within plan mode because if I didn't time-lapse through it, we were seeing some issues with this having like free model timeouts happening from within open router. I've not really seen that before. So, I'm just more focused on getting results here that are actually properly completing and we'll see what we get. Good. It didn't try to use ray, so we don't have to do any correction for it or anything. All right, we have a few compilation errors. pretty much to be expected, especially for a model around this size. The big thing to see is now if it fixes those. Very good. So, it did fix the compilation errors. Now, let's see if it opens up here. This is actually It looks like one of those like bathroom sign stick figures that was on the skateboard, but it actually was okay. So, now it's just let me rewrite the game with improved physics, cleaner trick logic, and more polish. In the meantime, I do want to just take a quick peek at what we have is sometimes doesn't always turn out that the second iteration run through is better than the first. This really isn't half bad. I don't know that this is like a California skate aesthetic. Um California boardwalk aesthetic, but spaces to ollie one to four is tricks. Okay, there's there's a bit of issue with the Olly logic, but it's actually like it's quite quite unique. This is better than some much larger models I've seen attempting this task. So, I have to say so far, just in the few tests we've run, I'm actually kind of bullish on this model. Oh, okay. There we go. So, that's it. just opened another instance of it that it itself is just verifying to make sure. Now we see NPCs. I want to give this something I' like I don't run normally just to see if the performance stays good. It it really like Yeah. Okay. It looks like a skateboard during like the big one, but aside from that, it it actually really fixed a lot of things. The ollie wasn't quite working before. Okay. Four is a 540 spin. Oh, okay. Oh, well, it gave us points for it, so the animation just maybe wasn't like Okay, kick flip. And look at the particle effects when we land. We do have NPCs moving around. It put trees in. It did put storefronts. The attempt at water was made to give this a boardwalk aesthetic. I think G was to get on a rail and grind. We would have to locate one without the vibrating environment impeding our ability to actually find it. Um, which may not be possible. I I could have sworn we just snapped to a rail, but I Yeah, I think. All right, I'm hitting G when we get to the center. Yeah, that's that's a rail snap. I go out on a limb and say this is a very it's actually passable. I think I don't know. Tell me if I'm wrong, but I think for a model of this size, being that I haven't really tested one in a while, I don't think the poolside model by Laguna, actually, no, not that I don't think. I know for a fact that that model did not do this good of a job or even anywhere near this in this specific test. So, keep that in mind as well. That's pretty much my closest reference point for this. Aside from like GPT OSS 12B, which wouldn't have done this. Now, we're going to give it the Subway FPS test. This is all just in a unified prompt. I want to just see what this pops out. So far, it's been good enough that I'm thinking we might actually get something playable here in the first try, or at least after a simple issue fix like what happened with the browser OS result. So, we're getting rate limited pretty hard right here. And something you'll notice when that error message goes away is the context length has really grown a lot here. Something I'm noticing that I'm not huge on is basically it will write the file. It will find there's some issues in it that it wants to fix. It will try a couple of times and be like this isn't going too well. I'm just going to rewrite the entire thing from scratch. And then that's happened a few times which can be equated to why the context length right now is 40% used up. But it does seem like it actually ultimately gave us something that is playable. So let's take a look at it. I have turned the speaker on. Okay. All right. You know what? This is It's not great. Overwhelmed. Yeah. Well, maybe if I could use the weapon, that wouldn't have happened. It's not great, but there's actually some cleanliness to the at least the graphics. I guess it's well, we have a mini map and that seems to show like the um I don't actually know what it is that that's showing. None of our ammunition is going down when we play. I think I'm going to give this an opportunity to improve this result. We'll see what it does with this. It's 50/50 that it makes it better or just ends up significantly worse. I don't know which one I would guess. still can't shoot, which is frustrating. It does seem like it fixed maybe some of the scaling issues of the station, as well as how many enemies there were. Those were two things I gave it as complaints. And it does seem like it actually handled both of those. The big problem, unfortunately, is that there still is no ability to shoot. Let me just do like a hard refresh. Yeah, left click is to fire. So, that's unfortunately just not working. But everything else here is actually it showed a decent ability to improve based off of feedback. I guess except for the one thing that I needed it fixed, which was the firing. I'll give it one more chance because so far I like this model. Also, no sound effects. We should now have shooting and sound all working. So shooting actually is working. I know it's kind of difficult to see, but the ammo is going down now. And that wasn't happening before. Let's see if we can actually get rid of Okay, and reloading. See if we can get rid of any of these enemies. Nice. All right. Good. Good. So, the only issue that we're having here now is the sound isn't working. Let me just make sure like that's not on me. That was totally on me. The speaker was off. So, oh, that guy definitely Usually they fade out when they get destroyed, but not in this case. Okay. So, some of these issues may have been um a Bjan issue where like there was sound. This is actually all right, especially for the size of the model with this few active parameters. I'm okay with this and it's continuously fixed some issues. I mean, context used up right there 63% of the 256K and it was still competently fixing things. So, I like to see this. I want to now give it the 3D watch website test but here from within the chat interface in open router. So without the ability for it to first generate a script and then be like okay I want to fix a bunch of things here. I want to just see what it actually comes up with when given no opportunity to iterate past the first result that it thinks of. So this is of course the website for the slapest watch company. It should have a 3D model of a watch in the hero section, a cinematic panning camera shot and then additional like luxury watch aesthetic. All right. So, here's the watch website result just from the open router chat interface. Okay. I think what may have happened here is did I give it the prompt where it doesn't specifically say this needs to be a 3D model of a watch? Yeah, I did. Okay. So, that's on me. We'll look at it still, but I'm going to do one more where I run it with the properly fixed prompt. All right. So, we'll try one more. Just specifically mentioning it needs to be a 3D model of a watch, but it did have a cinematic panning shot and it actually pauses when we hover over it. I don't see any specific hand movement, but the tick marks are in the correct place more or less for the hours. There's a date thing. There's also the second, hour, and minute hands. Scroll down where tradition meets innovation. Okay. 48 artisans, artisians, no artisans. And then we have our different watches here. We have the Sovereign and the Horizon. The Horizon seems to have a bit of an issue with its manufacturing. And then the rest of this is kind of part of the course for what we usually see in this prompt. It's always like golden white and like elegance and high-end luxury. Here's the same prompt, but the watch needs to be a 3D model. All right. Unfortunately, it just disappeared. They're not showing up. And because this was done through open router, I'll give it one chance here to fix this because I like this model. All right. Let's see if that fixed it. Good. It did. Yeah. There's a a bit of weirdness here, mainly with the orientation. The straps are the wrong way, and so are the second, hour, and minute hands, but it did actually produce a 3D asset. It's orbiting around it in a Keyshot style manner. And then our collection is unfortunately still more or less 2D. But it was interesting that it was able to fix the issue and actually showcase what it did end up modeling for us. Cool. Let's give this a 3D modeling task. I'm giving it the one I normally do where I need a little 3D model of a V8 engine that will fit a 280TC motor inside of it. And it needs to be able to be 3D printed with few to no supports required. All right, it's going to at least attempt to write this. And this is something that I think maybe this would be a little difficult for a model of this size or performance, but I'm interested to see what it gives us. All right. Supposedly in like 90 seconds we have our V8 engine model right here. I'm just going to open it in open SCAD so we can look at it. Okay. It made something. The problem is I don't entirely know what to make of what we're looking at. It did include STLs, so the top half and bottom half. Let's look at these in a slicer program just to maybe get a better bit of insight. Okay, I can also tell you that's okay. So, that's the bottom half. Let's see the top half. Okay, it's I'm giving it a little constructive criticism because it did actually produce two clean in terms of like there's no weird floating overhangs or oddities to this pieces that would I guess hypothetically be able to be mated together. The big problem is it is absolutely in no way properly a V8 engine. Again, hypothetically, that was quick. Okay, so we just have this. Now, there are eight individual pegs. So, it's we'll just look at these in Cura, the slicer program. It made some changes. I don't know that this properly understands what a Van engine is supposed to look like, but it shows some ability to create STLs as well as an OpenCAD model. That's at least not any like weird floating artifacts or things of the sort. Although pretty poor in terms of actually what we wanted that looked absolutely nothing like a V8. All right, I'm now giving this the Chrono City prompt, which is what I've now dubbed this, where it shows the city over a number of different time periods. I've been now seeing this trickle on to X as well. prone thieves. And now this is something that I historically would never really give a model of this performance level, but it's been competent enough so far that I want to see what we get. All right, so here's our Chrono City result. I think it did it a bit quick. It was like 3 minutes and 45 seconds. Okay, it's simple, but it's actually it's understood the task. We do have moving cars. There's even a 1945 looking pedestrian based off of what they're wearing. The buildings are the density of these windows given the size of the buildings is giving me some level of discomfort that I can't accurately describe why. But the cars are moving on the street. They're just cubes. But all right, let's see if the transition effect is there. There is also sound. And it's got an orbiting effect. I mean, this really it's bad, but it's not that bad from like a Okay, interesting. Like a Runescape style sound effect between the two. much happier, much more colorful. Our folks are all in orange and have yellow top hats. The vehicles are also more vibrant. So, the entirety of this scene actually changed to reflect a more vibrant color palette of the 60s. It's going to be dark and neon. Yeah, it always is. And all models so far have basically done this same aesthetic for 2055. So, there seems to be some common are they wearing sunglasses? I can't imagine they would be, but actually that does look like sunglasses. Looks like a Next up, we have 2005. Okay, more blue. Is that a taller singular building in the center that was not there before? No, it was. Okay. Oh, they're gonna Oh, all right. Everyone is Oh, the cars have wheels on the bottom of them. They're just under the ground plane. Everyone is bald to such a degree that the sun is reflecting off their heads. Let's Let's just check 2025. Very green. Okay. Kind of the same thing, but a different color. The people are now like fully emissive in terms of their color. And then 2055. Let's see if the neon Yep. Okay. Yeah, that's a pretty common thing that happens. Did the building get taller in the center? I really think it did. So again, really like this is nothing to write home about, but like the core foundational elements are put together competently and this was the first try. There's a nice orbiting movement to them. It changes throughout. There are effects between the scenes, their sound, and it did some level of like reflection of the specific time period like the vibrant colors of the 60s or the neon colors of the 80s or 50s in the 2000 years. So pretty pretty not bad. All right. So, for the last coding test, I want to give this another C++ test, but one that I've given very, very rarely to any model. So, it's less likely to have made its way in the training data, I would think. But, it needs to generate a 3D racing game that has like low poly early rally game graphical aesthetic to it using C++ because it really did a surprisingly decent job in the skate game, all things considered. And keep in mind, models of this size are more few and far between. But really the recent comparison that we have for this is the poolside lagona model. And so far this is definitely outperforming it from coding specific tasks. Though the poolside Laguna model was like top tier when it came to roleplaying and creative writing. So keep that in mind as well. Having this and then that on a DGX Spark may make for an interesting pairing of like a creative writing and then a coding capable model. You could probably do some pretty creative things entirely on your local device with these two models as a pair. not running at the same time, but like swapping back and forth. It seemed like the game was running for longer than it had anticipated just doing a test run for. So, I stopped it and then gave it a description of the issue as well. And this doesn't seem to want to go away. So, all right. It was pretty confident that it was working, but now unfortunately, all right, good. It quickly killed the processes. So, just in terms of like helping do system stuff at a basic level, it's good at I guess. All right. Supposedly, it fixed the black screen issue we were getting. All right. It said it was going to rewrite it completely, and it did just try compiling it there, and there were no errors. So, all right. Supposedly, and keep in mind the context length here is now it's at 75% utilization. So, this rewrote the entire thing because it was just consistently hitting a black screen. Unfortunately, that is still happening. and I'm going to call it at this point because it's the context length is kind of getting up there and it also failed to fix this over a few different iterations. Now, the reason I wanted to do this is because it was doing a good job so far, especially with the skate game and I wanted to make sure when giving it something that was likely less prevalent in the training data, it would still do a good job. Unfortunately, that didn't happen here, but again, it could also just be a fluke. The model performed well overall throughout the tests really just in that one we failed to get anything decent and it didn't really do a good job of actually improving its result which it did for a lot of these other ones. Even starting with the browser west I mean when we first looked at this and like nothing was working I was like oh no it's going to be one of those videos and I was fortunately proven wrong. The subway station FPS it did a good job at iteratively fixing this. I think I gave it like three specific things that it needed to fix and it did do them all at varying levels of success. Our watch website was acceptable and then the city time thing, although simple. Basically, I think the culmination and takeaway of what I've noticed here specifically is all of these results are simple, but foundationally they actually seem to be somewhat competent and together. So, it seems like some of the core capabilities here are definitely there and actually competent. So yeah, it's not going to make like an awesome polish city version of this, but as a starting point, this is really quite acceptable. And again, I really think that this paired with theund something billion Laguna model that we just tested recently, I think that you would have a awesome creative writing model there and then you would have a decent coding model here. So these could be some interesting things for machines like this. So, I wanted to test this and it does seem like it is hypothetically going to become open weight soon just based off of what they've been posting on X, which would make a very very compelling option for those systems. I don't know how this would perform at a quantization, but that's always something that needs to be tested once it is released and out there in the open, but as a kind of first impression, I'm actually happy with this model. So, I wanted to do something. I'm a little tired, so this may be a little shorter and less like long than normal videos, but I wanted to test it. And seems like some more exciting releases are coming, so that should be fun. If you have any questions, please feel free to leave them in the comments. And thanks for watching.