Xiaomi Mimo V2.6 Is INSANE? – Pro & Flash FULLY Tested!

0:00 And already this is just such a significant improvement over what we got with Pro. Today we're going to finally be testing Mimo V2.6. Both the Pro and Flash models

0:13 will make appearances in this video. This is something I wanted to have out quite a few days ago. However, I encountered a number of problems just getting them to perform well. I ended up

0:22 swapping to a Windows system, which is what we're using right now, downloading their native harness, and the performance was better, but I was still experiencing some tool call looping

0:31 issues, things that I also noticed when first trying this on Linux through things like Pi, Open Code, and even the Codeex CLI. So, this is going to be a bit different of a video because a lot

0:42 of the results I have already run in the Quest to get a better feel for these models. I saw a ton of polarizing opinion on these and my best analogy for how they performed for me is almost like

0:52 think of a sine wave where like sometimes it's up and then it goes back down and then it goes up again like that. That's how the performance was for me. Sometimes it was fantastic and like

1:01 wow this could get insane in all caps in the title and then other times it was like what are you doing? So very choppy performance but we're going to be taking a look at these models. We'll start out

1:10 just by looking at their scores, some pertinent technical information about them, and then we'll jump into some I won't say fun and light testing, but some heavy testing. Now, in the V2.6

1:20 series, we actually had three releases. So, most folks are probably familiar with the Pro version and the flash version. However, there was also a version of Quen 9B, a dense 9 billion

1:30 parameter Quen model based on Quen 3.5, I believe. And this actually was imbued with distilled reasoning traces or whatever from one of the larger V2.6 models. So, this is actually kind of

1:42 interesting. Not something we're going to be specifically testing in this video, but it's absolutely worth just mentioning because if we see some of the scores right here in their improvements,

1:51 this is definitely worthy of like its own independent analysis, I would say. So, for the larger of the two models, the Pro model, this is a little over a trillion parameters. It's a mixture of

2:00 experts model with 42 billion active. It natively takes in basically every modality. So even video and audio which is not necessarily as commonly seen and there is a million token context window.

2:11 Both of these are on hugging face under MIT license. So there's nothing really like sketchy at least in terms of the license. It's not some like show me custom license which is always nice to

2:20 see. Additionally to that, we also have the flash model, which if we scroll down is something that I think would realistically be a good option depending on performance for like dual DGX sparks

2:32 or something of the sort. This is a 309 billion parameter mixture of experts with 15 active. It shares the same modalities, so video and audio, and it does also have the million token context

2:43 window. Now, we also of course have some benchmarks about these models. And I will say the flash model considering that it's like around a third of the overall size is within a stones throw.

2:53 Well, maybe not a stones throw, but it's within like a leap of the pro model, which is always nice to see, especially because this is something that can potentially be run on some hobbyist

3:02 level local AI setups. Additionally to that, I'm probably not going to pay too much attention to the rest of these because I want to get hands-on. However, something they did that was really quite

3:11 cool is they actually showcased the training run for this online and live. As they mentioned right here, they streamed this live where it's now stopped. So, this is probably just a

3:21 snapshot archive of it. But the entirety of the run, all the steps like checkpoint times, the total cost, and things of this sort were actually showcased live online. something that is

3:32 very interesting and a look into something that I don't believe any other lab has really this prominently displayed. At least not ones as large as Xiaomi. So, this was very cool to see.

3:42 Now, pricing here is a big story. Not to sound like an AI, but like this is a pretty loadbearing price though. So, the flash model, the smaller of the two is 14 cents per million in and 28 cents per

3:53 million out. Extremely cheap. Pro is 43 cents per million in and 87 cents per million out. Additionally, extremely cheap, especially considering some of these stated performance numbers. Now,

4:06 in terms of how we're specifically going to be testing this today, I did actually pay just myself because I like to just get what like the average person will get. And if you get like special access,

4:16 you never 100% know like is this what someone buying this randomly would get as well. So, I believe the plan I purchased was $44 a month discounted from like 50. And we can see right now

4:27 I've currently used up 31% of that plan. And that is a monthly plan. So, that's um I don't know what to make of that. I've run quite a few tests, but I don't know that this usage figure really

4:40 reflects what I would have expected. It seems a bit high, truthfully. However, it was doing a bunch of glitching out looping tool calls infinitely. So, that could probably contribute to like a

4:49 billion or so tokens here. Let's now go ahead and swap into the desktop app where we're going to see initially a lot of results have already been run. So, this first one we see right here was

4:57 done with Mimo V2.6 Pro where I made a huge mistake because I intended to paste in the prompt for the Guitar Center game where you go around and you can play instruments and then get into fights.

5:09 However, unfortunately, that text file also had a separate prompt that was run with GPT6 Soul to make a showroom website with a 3D model of a Saab 900 Turbo. It basically just did both and

5:22 made them separately. It worked worked for like an hour and 6 minutes. That's on me. My bad on this one. I'll own this. However, we basically have something right here to take a look at.

5:31 And like I said, this video is going to be a bit different than normal because I needed a lot more time really to test these things and see how they do. So, here is Guitar Store Shred. Now, this

5:42 was done with Mimo V2.6 Pro. I have not looked at this yet. And had I looked at this, I likely would have decided not to show it to you as the first thing. However, I suppose what we're going to

5:52 see right here is a ah okay. So, I believe that the big issue right here is it's probably drawn the full scene. The camera is just basically over the roof. So, we have no ability to actually see

6:06 inside the store. Unfortunately, I'm not noticing any way to actually adjust the camera. However, we're going to definitely need a follow-up to the guitar store. So, let's just initiate

6:17 that now. And in the meantime, we'll go look at some other results that are finished. All right. So, I've just told it the guitar store camera is above the roof. We can't see anything. Our guitar

6:25 store simulator has been fixed. The camera is no longer above the roof. Okay. It seems simple, but Oh, yeah. This is All right.

6:44 I'm just I'm I'm going to be mature about this one. All right, let's try to initiate a fight as the store rage indicator is rising. Uhoh, I think we just got attacked.

7:09 Yeah, we did. Our HP is going down. All right, let's take care of this aggressor. Oh, yeah. See you. Now, I think if that gets to 100, the entire store riots. So, let's just try

7:24 to get it there as quick as we can. All right, that was The problem is we've seen what Opus 55 did for this prompt. And the problem also is that one was just bad. I'm going

7:48 to be honest. That was not very good, but it also could have been partially because I accidentally sent it the double prompt. So, that is perhaps on me. That was a pretty good introductory

8:01 experience, I suppose I could say, for for this. Now, I don't know that this one is running as well. We'll just try it. This next one is, of course, and I will go over the prompt just a bit more.

8:11 This is it needs to make a 3D model of a Saab 900 turbo, a mid80s one in Blender, and then build a showroom website for it as if this was a brand new car that had just come out, so folks could go in, see

8:23 different areas of the car, and then basically look around in it. Now, I'm going to notice this is not a very good model, just at all. It did get some of the sloping rearward shape of this

8:35 vehicle and it also does have some specifications here that are actually somewhat in line with the realistic vehicle. Now, we have different trims. Okay. And it just seems like it changes

8:47 the color. All right. We can open doors and things like this. Okay. The passenger door, the hood. Uh, I think we're all going to pretty prominently see that the biggest issue right here is

9:00 it doesn't seem like it has any detail that's visible inside the vehicle. So, so far definitely not off to a great start. And I will say just in some of the things I had seen, I found this Pro

9:13 model to be significantly less impressive than the Flash model, which for a local AI enthusiast is probably good news. But it was still like sit inside

9:25 reset view. And then we have just some other wheel styles which unfortunately not 100% work. So yeah, that's um Oh, I did not even realize there was more to this.

9:38 Wonderful. Engineered by people who build jets. That is true. We do have some more specific specifications here. I'll say the front end is not bad. A cockpit, not

9:49 a car cabin. Okay. giving like some arrow inspired things being that Saab they say like born from jets. I mean they're out of business now but and then we we have some like glamour

10:01 shots of the vehicle from Troll Hutton. That is true. That is where they were made. They do have a SA museum there which is sick. And then we also have some

10:11 Hey, I've seen that car. That's pretty cool. Okay, good. So it has some interesting pertinent lore and information about these vehicles. And then we have an inquiry. I'll say the

10:19 actual website here, if we ignore the models of the vehicle, front end wise, this definitely is not bad. And this hero section would look really good along with this site were this model

10:29 actually like good. Now, let's take a look at some other things. I don't really even want to do a browser OS because I saw a lot of folks saying this is benchmaxed. I don't know if that's

10:38 true or not, but the browser OS is not going to really give us any insight into the performance of this model. I mean, maybe we'll try it later, but this right here is something I do want to highlight

10:48 because this was quite impressive. I gave this a markdown file and it says 9 hours and 26 minutes. I don't I mean, I want to I went to sleep, so maybe. This was something that in my most

11:03 recent video on Klein, one of the tests we did was actually give this markdown file that was generated by Fable 5.1 for a retro Formula 1 style game in C++ to a bunch of open models and see how well

11:16 they did. One of them was GLM53 and it was significantly outperformed by this model right here. And this is the flash version. So, if we actually look at some of the screenshots of what it

11:28 produced for us, this is more or less like a pixel perfect 95% there replication of the source game. This was really truly very impressive

11:38 and something that made me like, wait a minute, this is a very, very stark display of intelligence right here. So, let's just open this in a folder and check this out.

11:50 So, this really did an absolutely exceptional job with this, the vehicle selection screen, everything like that. If you did watch my previous video, thank you. And you can see how some

12:01 models approach this and it was nowhere near this performance. So, let's just Okay. Uh, I'll go with that. All right. We have different camera views. We have

12:14 sound. We have the steering telemetry. Okay. So, The one issue I'm noticing that you're probably going to have picked up on as well is it seems like the position the

12:26 car started in moves with the vehicle. So like this or there's a shadow issue there. However, like look at the actual flames coming out of the exhaust when we like and the thing is the way this is

12:39 actually spinning out is realistic to how some of these cars from that era actually performed and it just kept that logic in from the Fable 51 source which is cool to see. So, oh,

12:52 okay. And and like these are basically uncontrollable, which is kind of how they are in like much better racing simulation games as well. So like

13:06 it's just kind of it was well done. And I have a couple other reference points to how other models did with this and this did a significantly better job even with the cinematic camera and things

13:17 like this. So and we can see even the NPC cars crashed cuz these things are just downright uncontrollable. But that's cool to see. So, this was something I saw that I was

13:32 like, "Hold on a second. This is actually quite impressive the way it did this, especially because one, this is the flash model, and two, this is something that I've recently done. It's

13:41 a gigantic design markdown from Fable 51, and GLM53 did not do as nice a job as this did. So, interesting to see. And a little more bullish, at least for the flash model." Now, this is one I have

13:54 not even looked at one bit. So, the first things we see right here are going to be my first look at this. This was using the Pro model, and I gave it a prompt that was given to a GPT6 Astra

14:06 versus Fable 51 test, I believe it was, where it needs to create a Hot Wheels I'm a I gave it the wrong prompt. I had intended to give it a prompt where it needs to create a simulation of a Hot

14:19 Wheels crisscross crash track. I accidentally gave it the prompt that just says, "I want a 3D printable model of a car that will do well on this track." However, we now have something

14:29 to view. And I mistakenly Wait a second. Oh, yeah. Yeah, I messed this one up big time. This just kind of it just encapsulates

14:46 like this, but this one was just so I was like throwing things at the wall. to see what stuck. Now, okay, we're we're going to have some renders right here of what it made. I will say I don't know

14:57 that this is the most optimal shape for that track, but this is definitely a lot better looking than what GPT and Claude had made. They just basically made flat wedges. A big issue here is of course

15:08 the floating spoiler in midair. However, we do have like some bits of information. We have some GDAU shots as well because I mistakenly said make this using Blender and GDAU,

15:22 assuming it was the full track prompt and not a mistake. I do have to say though, I don't know like where the wheels would go because I mean,

15:34 let's just call this one a mulligan. Now, for a quick example of like where things were going wrong, we can see right here, I don't actually know if I had initiated this with the flash model.

15:43 I may have changed it after the fact, but let's just it doesn't matter. This showcases one of the big issues I was having and even in their native harness where I gave it a prompt right here, a

15:53 text file just for the Chrono City and unfortunately the location that that was in got removed. It was on a flash drive when it was trying to read it and it basically just went crazy looping it

16:05 here and it was really quite frustrating. We can see coming up there will be some follow-up messages from me being like, "Are you kidding me? What are you doing right here?" like question

16:13 mark question mark and things like that. Something that is on me for having removed the media that had this prompt on it. But the problem is the model just like kept trying to find it. I feel like

16:26 a model of this caliber should have probably been like hey I can't access this file location like can you please ensure it's there rather than just go into a loop like this. And this was

16:35 something that was kind of a bit of a let down. I then pasted it back in when I realized it was on the flash drive and I said, "Copy it into your folder and then read it." And unfortunately, it

16:45 just this thread was more or less like cooked in my opinion. So, I ran it again in a new one again with the flash model. And this is the Chrono City result. So, I just started it by saying, "Do you

16:56 have this prompt? Then build it." And it worked for an hour and 20 minutes. And this is the flash model. All right. We'll just look at it in the app's own browser, I suppose. All right. All

17:05 right. And we see right here, this is our 1945. Definitely not bad. And this is the flash model. So, I've seen models that are larger than this and closed source that have not done as well with

17:15 this. So, keep that in mind. I wonder, can we move the camera up and down at all? Oh, no. But I accidentally initiated one of the um timeline shifts there. Let's just look at 1945 a bit

17:25 more first. Keeping in mind, I don't have up and down camera control, so we're kind of stuck with this. Interesting. There are buses. The folks are in older style clothing, I guess

17:36 could be said. Interesting buildings. The trees are geometrically pleasing. And then there's like an old-looking vehicle. We have city storage. Are the street lights changing? They are. Okay.

17:48 Good attention to detail. And then we have like the water storage tank or whatever that was on the top of the building. Even a fire escape. All right. Simple though in just a little four

17:58 building city scene. Let's now go to 1965. So, I like that the way that it actually uses that like plane to transition everything I think is well done. Good.

18:11 The vehicles have fins. The people are in far more flamboyant and happy clothing. Additionally to that, I don't actually remember off the top of my head if the buildings changed at all in

18:20 stature, but we can definitely see an overall change in the aesthetic of the scene here. Just colors and things like that. Definitely more reminiscent of like mid-century style in the '60s.

18:33 Did the buildings change at all in height? Okay. No, they didn't. But that's pretty cool. I like this angle right here. Next up, 1985, which most models just make like purple sky,

18:44 rain, and like orange tinges. Okay, so purple sky. Yes. A lot more neon. Arcade fever. Did the face of this building over here change entirely? It did. I like that. Good. To a more 80s

19:00 look. And I actually think this like glass design is fairly 80s. So I will say attention to detail on that one was actually somewhat respectable. Did the street lights change at all in design?

19:14 Okay. No. All right. So let's go from 85 to 2005. Okay. iiggate.com. I don't know what that is and I don't

19:26 want to go to a random domain, but igate.net. net. I just I'm going to just basically ignore those. The vehicles are perhaps a bit more modern looking and everything

19:38 is a bit more toned down. Did the street lights change now? I think they did. Yeah, they did. Look at that. Good. Good. These are things I like to see just cuz it's more attention to smaller

19:49 detail. All right. Then 2025. Hold on a second. I saw like a few things that also changed it. That looked like a goose for a second, that

20:05 car right there. But the vehicles are far more capsule-like. They look they look interesting. Additionally, let's go over here. So, there's a helellipad on that

20:19 building now. Was that present in 2005? No, it wasn't. and the AC units were removed and swapped for solar panels here on this building. So, some of this detail and

20:30 attention to it I'm pretty pleased with. And again, this is the flash model. So, this is the smaller of the two. Finally, we have 205, which usually ends up looking like 1985 for some reason. Yeah,

20:41 I mean, personally, I dig this. That I have no idea what is going on there. Why are they okay? I think those are drones or or hover pods or some form of thing. We have helix labs. Let's see if

20:56 the roofs of the buildings have changed at all to reflect this time period. Skyane open. Fly quiet. Okay. No, but the lighting here is actually kind of cool. Look at that. Like,

21:07 we'll compare it to 85 cuz that's the only other one that has like kind of dark out. That's cool. I'll say overall this was acceptable, especially for a flash model

21:18 of this size. And the attention to detail and some of the smaller, less noticeable things that were also changed speaks more to me than it just being like a simple fourb building city block

21:30 scene. So, I do like what I see here with this result. Next up, this was one where the pro model really, really showed itself in a nicely capable regard. This is something that I first

21:41 did for the first time ever with GPT6 Soul. I said, "Create a pixel perfect replica of player versus player in 2007 Runescape at the Grand Exchange." It gave us an initial result that was kind

21:53 of mid. It was basically like 2D and I wanted 3D. I even did I did not specify that. Okay, my that's on me. But I also had this issue with GPT6 Soul where the first result was

22:05 like eh. So I had a follow-up sent and that's what I did here as well. So, the thing it actually created was like, hold on a second. This is sick. And I will say this is better than the result that

22:15 we received from GPT6 Soul. This is the pro model. And both of them needed a follow-up prompt to make it better as well. So, this is something right here where it's like, huh, maybe this could

22:24 get insane in the title cuz like this is cool. And the thing is like it even has the like walk here in Runescape. So, the squares we have different like options and things. Now,

22:37 I truthfully don't know 100% how to like uh initiate like combat and things like this, but it did a I was satisfied with the way this did.

22:52 Maybe it I don't know. Someone would have to kind of help me decide if this was better than what five six or what six soul did. But like the mini map, some of the inventory, the sharks are

23:02 definitely better than what soul did. I do remember that. Fight again. And we have equipment, we have prayers, stats, and magic. So, we're full out maxed. We have 99s across the board. So, we should

23:15 really be smoking this guy. Unless he also has vengeance. Look at that. So, these are things from the actual game. It's ice burst. Come on. Freeze him. Freeze him.

23:27 Fire strike. Oh, okay. Fire strike was very flamboyant. Fight again. Oh, we just potted up. And it even says you eat the shark, it heals 20 hit points. Oh, yeah. This guy's

23:40 going to get slapped now. I don't know what that is, but Oh, a fire cape. Look, look. Is it actually If it has movement a like borderline. All right, we need to continue. Oh,

23:51 attack. Okay, so we have to click. Did he just safe safer? Oh, that's But yeah, this was I liked this. I found this to be quite impressive. He's safing. So, I'm going to safe, too.

24:12 For those who are not um aware, safing is when you eat, even though your hit points like aren't at a low enough level where your opponent could really easily or realistically get a kill on you. So,

24:23 all right, buddy. You want to play that way? Let's go. I want to put my fire cape on. I'm gonna just start. Oh, cool. We equipped a uh a sword.

24:34 Is that an abyssal whip? Yes, it is. Well done. I think we might actually get this guy now. That's

24:48 This guy's a So, yeah, this was one that I was pretty pleased with that I wanted to showcase just cuz I thought it was kind of endearing, perhaps nostalgically based, but I liked it. And that was

24:59 again the Pro model. Then again, I truthfully have no idea what even this was, but this was the Pro model. And again, just to showcase some of the issues that were being had with this in

25:10 its native harness without like anything special put in, it's just looping like crazy. And finally, I found it and I I got mad. So, we'll go find that. And this was the Pro model again. Have you

25:22 made any progress? And it just started looping again. I yelled at it. I said, "WTF? Are you unable to see the command history and that you have looped trying

25:32 to read the document like 50 times?" And then I just gave up. So, oh, this was the retro F1 arcade game. So, this is when I was trying to get it to produce the result that we saw here, the

25:43 replication of the fable design markdown, which ultimately got built by Flash. So, I'm almost happy it failed with Pro because Flash did such an exceptional job. We otherwise may not

25:53 have been able to see that. All right, so we're going to be trying that Guitar Store game again, but with the Flash model. I deleted the thread in which it had created it with the Pro model, which

26:02 was also with that Saab result. And I also went and deleted all of the results off of the disc. So hopefully it's not like got any insight in its history because that game was not very good. All

26:12 right. And already this is just such a significant improvement over what we got with Pro. Now, the argument could be made that part of the issue was because the pro result was run with the double

26:24 prompt. However, still like this is just so much better than like I mean it's Oh, Jada to swing. I was wondering Oh, yep.

27:06 Wait, I didn't even Oh, I had didn't matter. I hadn't seen that person there. A mistake. later. So apparently K is for a heavy hit and

27:35 still this is this is much better. Even the guitar models and things K is heavy hit. Why isn't it landing? Oh, it landed.

27:54 We have flying V guitars. And keep in mind, we're kind of stuck with this camera angle, but this was definitely Okay. And the guitar does come off the wall

28:05 when we pick it up. So, we even have some amps right here. Oh, and we're supposed to demo all of these. All right. That's That's okay. I just wanted to see this game.

28:23 One more. One more. Come on, Jay. Oh, yeah. For a flash model, I'd say this is actually quite good, especially Oh, okay. Right as I said that. Come on.

28:40 All right. Well, that was a nice improvement over the um Pro result. So, next up, we're going to take a look at one of the earlier results that I had run. This is with V2.6 Pro. It's our

28:52 traditional Subway FPS result. It had created something. Honestly, I don't even really remember at this point. So many models have been out recently. I said, "Just finish the result and I'll

29:01 tell you if it needs anything." And then I told it, "This is low fidelity and bad." It was trying to just troubleshoot the result basically once it had done that. So, I stopped it and was like,

29:10 "All right, okay." Yep. So, here's the result we received for the Subway game. This was again like not a fully finished one. However, it gives us just some insight into some of the issues we were

29:21 noticing where the tiles like nicely done, the bloom, the lighting, and things like that is just absolutely garbage to the point where this is definitely a prime one for a proper

29:31 follow-up. So, I'm going to initiate that right now, but I do want to just see the train come into the station. And so we have more feedback to give it. And again, this is the pro model that has

29:42 produced this. All right. Now the train should be coming into the station. It has frozen up. That happens quite often across many different types of models that when the train comes into

29:53 the station, they freeze up. So okay, creepy in a good way, I should say. So, we can see if the lighting was better and and the enemies had a bit

30:06 more fidelity, this would definitely be something respectable. So, I'm going to tell it it's a 6 out of 10. It can easily be an 8.5. Make it better. And I've given it some of the specific

30:16 things I did not like, like the lighting, the enemies are too simple, and there are no bullet holes in the environment. Something nice to see is that in a little over eight minutes, it

30:24 made these changes. Now, we have not yet seen them, but for the Pro model, that is definitely a pretty rapid pace that it did that, which makes me happy. So, let's see if it really did bring it

30:37 from a 6.5 to an 8.5. Okay, the lighting is definitely less egregious. And there are now bullet holes in the environment. So, the two main things that I wanted fixed that were almost kind of like bugs

30:51 were actually fixed. Ah, the third. Are the enemy models better now? Yeah. I mean, definitely. They're less simple. Before they were just super simple shapes now. I thought that thing

31:07 on the ground was like a health pack, but maybe the green one is. Oh, yeah. Oh, come on. You feel Oh, that's fell. Well, you know, I'll own up to my

31:25 mistakes. Oh, okay. This one. Ah, they are backwards. So, they did have faces and things. Good to know. Oh, okay. Goodbye. The train has now arrived. If it backs

31:37 out of the station, that's like just not it. Sometimes they do that. Sometimes they continue forward as they should. This one's staying, which is a neutral take on whether or not that's good or

31:51 bad. Some of the lighting is actually cool. Like the green, it definitely makes it creepy. Okay. And once again, we feel we fell. Not bad. That was definitely an

32:03 improvement. And in only 8 minutes, so I am happy to see that. Although, it's still a far cry from what we see at like Frontier level. Now, I also gave the Pro model the self-contained C++ gate game,

32:14 the one where it can't use RayBib and it has to be a New York City map. It only worked for around 28 and a half minutes, which is truthfully rather quick. So, if we go and look at this in its folder,

32:26 we should be able to get a look. And this is one, okay, I I didn't like have a full recollection of how this looked. There are things here that I'm going to say are good and then things I'm going

32:35 to say are bad. So, something I do think is more in line with what we see from higher quality models is some of the walking animation here of this figure. Ignore the fact that the head is like

32:46 halfway into the chest. That's something we see done better on larger, more performant models. Overall though, this is still just a massively simple simple take on this. And I feel like it should

32:56 be so much better and it's not. So again, something that I almost have to wonder if the flash model would do a better job at this. And it's one of those things that I'm curious.

33:09 The board does move independently from the player, so that's good as well. And just another thing that denotes a model that's thought it through. We do have the fire hydrant spraying which is a

33:20 very very commonly understood thing for a New York City block at least in whatever these things are trained on. Yeah. So this is again like simple but it's just it ain't it. I want to try

33:33 something. I don't know how this will take effect but I've swapped it to flash. This is mega mid. So, we did swap this to Flash

33:44 and have it fix the issues with the skate game. Now, interestingly enough, it decided that the current build was too plain, which yes, it was, but it decided to just completely redo it from

33:54 scratch, rewriting the game from scratch with a much higher visual and audio bar, textures, sky, shadows, sound effects, and a cleaner pipeline. And it did just as we saw, smoke test the improved

34:05 result. And just on first glance, it did definitely look better. Let's take a look at our updated version. It was the one that was based off of what the Pro model had made, redone by the Flash

34:15 model, just in case you uh skipped to this part of the video. No, all right. Good. We do have some sounds. And it seems that Oh, can we hit pedestrians? All right.

34:33 It doesn't seem like we can. B is for wireframe. Oh well, B. Okay, maybe that doesn't work. It's simpler than I would have liked, but

34:58 keep in mind this was a bit of a non-traditional way to go about this by having the flash model fix what the Pro model had done a mid job of. So, famous pizza. It's interesting how the store

35:09 names actually are not locked in one place. They move depending on where on the map you are. Yeah. So, overall it's it's it's like consistently inconsistent. So,

35:23 overall, that's going to conclude our first look and test of the Mimo 2.6 models. I would say just as a probably simpler summary though, I do want to actually just look at a final tally for

35:35 how many specific tokens were used in this plan. a little over 12 billion and that accounts for 32% used of a $44 a month subscription which I think was discounted by $6 so of a $50 a month

35:47 subscription. I honestly think in terms of a takeaway I believe I said earlier it's like consistently inconsistent. So sometimes it's really quite impressive and then other times not as much. I

35:59 would say overall if I had to pick a winner here as to which of these models I was personally more impressed with in the testing that I've done so far, I would absolutely go with the flash

36:08 model, I think it really showcased some capabilities that put it in a like pretty decent space, especially considering its size of around 300 billion parameters. the C++ test where

36:20 it had the markdown file from Fable 51, it outperformed GLM53 on that task pretty heavily. When given basically the same very intricate 20,000line markdown file with all of the design choices and

36:34 decisions made for it already, just basically build this app and it did a really very impressive job. The Pro model is harder to judge because I did not have as nice of an experience using

36:45 it. I noticed a lot more issues with the Pro model and that doesn't mean it's bad. It means it performed poorly for me in these specific tests we were doing. I don't I mean some of these 3D results

36:56 here, we saw the Saab 900 model that the Pro model had made. However, that may have been a little contaminated because I accidentally ran those prompts wrong. But the Flash model, like the Guitar

37:06 Store it did and things like that, it seems to actually be pretty good. So, my takeaway from this is the Flash model might be insane, but the Pro model might also be insane, but not in the way that

37:18 would go in the title in all caps. So, I don't actually know what to title this video. However, I wanted to test these. I know a ton of folks were interested in seeing them, and I saw a large amount of

37:28 discourse online, just people saying, "This is benchm, it's garbage." And then some folks saying, "I've had a great experience with it. It's been really good." So, I wanted to get some

37:37 hands-on. an overall TBD I would say. So that's going to conclude our first look and test of the Mimo 2.6 family. Again, there is also that distilled 9 billion parameter Quen,

37:49 so just make note of that because that's something interesting as well. If you have any questions, please leave them in the comments. And thanks for watching.