Step 5 Preview First Test – Is THIS a DeepSeek Competitor?

0:04 That result right there was $6.36. Today we're going to be looking at an upcoming open weights release and that is step five preview by stepf fun AI. Now this model you may have heard about

0:16 recently because it's actually been out for a bit of time. However, the open weight release is going to be on October 15th which is about 3 days from now which they just mentioned right here.

0:26 Now, I had tried testing this when it initially came out, which I believe was a couple of weeks ago. Unfortunately, it was too difficult to find a reliable way to actually test it for agentic use so

0:37 that it wasn't timing out every x number of calls and things like that. Fortunately, now I think it's currently available for free in like open code. I'm going to be using it for today's

0:47 testing just through open router setup with the Pi harness. But I'm very excited to actually be able to get some hands-on testing with this model. So before we get into it, do feel free to

0:56 subscribe so we can get that 100K plaque, but also we're close to 80K, so that's acceptable. And let's take a look at step 5 preview. Now, Step Fun is definitely not as wellknown as folks

1:06 like ZAI or Deepseek. However, they have recently had some pretty impressive releases. And the course of their models evolving have definitely shown some pretty impressive gains. So I do quickly

1:16 want to direct our attention to their previous model. And this is not the predecessor of the one we're looking at, but it was an Apache 2.0. know openweight drop step 3.7 flash. We

1:27 tested this on the channel and it was actually quite competent especially considering size-wise it was on the smaller end of a lot of the flash models that we're seeing now. So I do just want

1:37 to quickly point this out and I am hopeful that the step five preview weights will be Apache 2.0 as well as the 37 flash ones are but we won't know I guess until October 15th. So step 5

1:48 preview is a larger model at 600 billion total parameters. It is a mixture of experts, so it has 27 billion active per token and it has a 1 million token context window. It is also multimodal,

2:00 so it can see images. Now, in the benchmarks right here for this model, they do show it as stacking up very favorably compared to some much larger models like Kimmy K3 or some closed

2:10 source proprietary models. Although Opus 5 is now kind of an older model, but when this was released, Opus 5 was still the newest Opus. We now have 55, which is quite something. So, I don't really

2:22 care much for the benchmarks really, though. I want to get some hands-on testing with this and just see how it performs. They do also give some demonstrations here just of it making

2:30 fun things in 3JS and a bunch of other work as well. Some more emphasis on longer running agentic tasks and some more technically impressive things like a GPU kernel optimization to like

2:41 improve speed up and throughput and things like that. things that we don't necessarily currently test on the channel, although I suppose there could be a simplified version of this in a

2:50 future testing suite, just seeing how capable some of the new models are getting. So, in terms of a bit more technical information about this model, we can see in the documentation right

2:59 here, it is not only able to see images, but also video as well. So, this is properly multimodal, and that's cool to see. It has a 64k token maximum output length, and then of course the 1 million

3:10 token context window, which we did see before. Additionally to that, if we scroll down a bit right here, we'll be able to get some information on pricing. Now, this is not ridiculously cheap. We

3:20 can see that this is a dollar per million in and $2.70 per million out. Because of the competition that has come with GPT6 Luna and then also Claude Haiku 5.5, it's getting harder to

3:35 compete in a price range like this, which before would have been thought of as relatively cheap. But now there are such high-end competitive models around this pricing that there's a lot of

3:44 option and I suppose as the person using the model, that's good to see. So we're going to begin with our browser OS test. However, I do want to point out a couple of things. one, we're using this with

3:54 the PI agent harness and I just have it set up to go through open router. So, we will actually be incurring cost for all of the tests we're running today. And at the end of the video, we'll just look at

4:04 the total amount we spend for all of the results that we see. Additionally to that, I have all of the tests. It's going to be set on the high reasoning mode so we can see the maximum potential

4:14 performance of the model. And with that, we have our browser OS all set. So, let's take a peek at it. So, here is our step five preview browser OS. Okay. Aurora.

4:23 That's not bad. Any password or none? Password. 1 2 3 4 5 6 7 8 9 0 1 0. Good. I'm usually good at guessing those. All right, we have some notifications in the top right. Okay, we can't I'm like

4:40 just used to going like that on Mac OS because these really infuriate me. The big one. Is there a right click? Yeah, there is. Okay, and we have the option to change some wallpaper there. This is

4:51 a very uh heavenly looking wallpaper is the term that comes to mind. We do have the date and time in the bottom right. Uh it gives us the option to like hover over them, but nothing happens when we

5:04 click. And then just some fake things one would see were we using a laptop or a real computer here like the sound adjustment etc. Oh, time machine. Oh no, it's recording. So okay, interesting.

5:17 And it seems like we have a few apps already open down here. Let's take a look at our start menu. Good. Grand Theft Aurora. Let's just check it out.

5:30 Okay, this is actually it was promising for like the first fraction of a second when we loaded in, but now it's just a bit something seems inverted, I guess.

5:46 Yeah, something's definitely inverted here, which is a shame because otherwise I think this had some pretty decent potential. Nonetheless, that would be a failure, but it is a preview version.

5:55 So, okay. Oh, look. We can actually move these around. And for some reason, they seem to like I don't want to move this now, but it stays there. Oh, right. Not bad. Don't trash. This is a good looking

6:10 UI, actually. Some of those photos are from this wallpaper. recent files, downloads. This is a pretty full packed file browser. I definitely see some Mac OS

6:21 inspiration for this. Although some of the window movement is not enjoyable. Like it's just snapping to our cursor and then it like moves around based on the cursor. But I'm just trying to close

6:32 it and it's somewhat Mac OS inspired. These windows are like theme Not bad. I haven't seen that. Like you'd think

6:48 we'd see that more often and we don't. No fetch. Neo fetch. Not bad. It looks like a dancing stick figure or a a happy stick figure.

7:03 All right, let's check out our next game. Again, I think the actual ship is like inverted, but this is also one that actually has potential.

7:13 I'm just noticing first impressions here. I do worry that there may be some spatial issues we notice in some of the later results with this model, but it seems to have like some basic sense of

7:23 like game design cuz that and the GTA game looked acceptable. Notes: All of my best ideas happened at 2 a.m. and none of them survive 4:00 a.m. Okay. And this did just get opened right around 2 am.

7:36 Snow crash. Funny. Well, not funny, but like it's a interesting book to reference in this task. It's fitting settings. All right, let's take a look at some of these other wallpapers. Oh,

7:46 this stupid menu. Like, okay, that's a nicely done Neon Horizon. It's hard to describe the issue with the window movement. read the full documentation. Okay,

8:03 that's our about I wonder if that's and then oh it's got like a snap feature like look at that. That's actually not bad. The final thing I need to get out of this like it's just the icons and

8:18 window menus are like magnetic. So if the mouse hovers over them, they get moved where you move the mouse and it's just this is not good. I'm not clicking on these and they're just Okay, let's

8:29 just get this over with. Here's 2 am. Zero windows. Zero windows. Zero windows. Okay, good. And then it returned us to 2011.

8:40 Okay, overall this was acceptable. I see some things that are giving me some hope and then some others that are giving me some stress, like this window movement glitch. All right, not bad, though. So,

8:51 the next test we ran was the C++ skate one. I ran a couple of these just in advance to ensure that it was reliable enough to test and I'm happy to report it was. And this is the self-contained

9:02 C++ gate game. The one where it needs to have the New York City block and it cannot use Ray Lip. So, let's take a look and see what this has made. Okay.

9:15 Oh. Oh, okay. It's troubled, but I see elements of this that are good. Like the fact that these skaters

9:29 built in individual pieces, kind of like what we see with the Opus 55 result and things like that where it's not just one static like humanoid figure. It actually has separate pieces so it can

9:42 hypothetically be moved. There's a news stand. I will say though, there's definitely a lot wrong here. We can bail. The text is also one of those things, but it does have the fire

9:54 hydrant shooting water. These are what I believe to be attempts at vehicles. Perhaps a taxi here. Oh, okay. And we have pedestrian. Can we interact with the

10:05 pedestrian? I don't think we can get over there. All right. And that leg is we are confined to this inner space. This is, you know what it looks like? It looks

10:21 like a photo you'd see in like a game asset pack where they just place all of the assets next to each other in one scene like this so you can see all of what you're getting. That's kind of the

10:30 vibe that I'm getting here. So, it's got few good elements, more bad than good, but I also feel like you can get away with a lot just calling a model like preview. So, I may have to use that

10:43 tactic one day if I rebuild something and be like, well, keep in mind it's still in preview. Let's try one more like grind. I don't think we can get up there. All right.

10:55 Yeah. Okay. All right. Interesting. Next up, we're going to be trying the subway FPS test. This is the newer version where we have to get on the train to go to the

11:06 subsequent waves, assuming we pass the first one. And this does say use 3JS. Okay. Yeah. So, we are going to see some of its browser 3D chops or perhaps lack thereof. So, we'll see what we get. All

11:18 right. We have our Subway FPS titled Subway Outbreak. Okay. Um I think my speaker died, so that's I need to plug it in. All right. Let's take a look at the H. That is really

11:33 quite frustrating because I really do see some decent potential here. The big issue is unfortunately, as we can tell, we're stuck under the floor and even the enemies don't actually look half bad

11:47 considering this is all just done in 3JS. They're not just super simple shapes put together in a humanoid like style. Really, one of the more frustrating things here is that this

11:58 took over an hour and just did not get us to a decent point. So, after a little bit of time and a power nap, we have an updated and hopefully fixed subway FPS.

12:11 Okay, better. Now, I do need to unmute the tab. There's still definitely some of the same sort of issues we noticed with the Skate FPS where there are strong

12:22 elements here. However, okay, cool. We can zoom in. There are also some not quite right like geometric things. Oh, all right. But as we mentioned earlier, some of the

12:36 actual models are not bad. They're not just really simple capsules. Ah, and the train should be approaching. Oh, yes. Okay. Nope. Um,

12:52 F is to board the train. Okay, I'm pressing F. I think it used like the same material from here for the train, which is We do have ammo crates that glow and we

13:06 can pick those up. There are, of course, no bullet holes in the environment. Actually, there are, at least there. Interesting. All right. Unfortunately, we're going to

13:16 have to stop it here because the logic to get in the train does not work. So, that is a little frustrating. However, we got to see some more of this game. And again, there are elements here that

13:25 are decent. Look at the rat. That's like stuff like that. And then elements that are quite troubled. So, next up, I gave it a Blender and GDAU test where it had to recreate this image in a pixel

13:38 perfect way, making a demolition derby game that we can play. it would have had to create these sedans in Blender and then the game in GDAU. We have the result here. I've not taken a look at

13:47 this yet. I am wondering if we're going to notice perhaps some spatial issues like we've been seeing. That's kind of been a common theme, but Oh, okay.

14:07 We have different camera views. attempts were made. Um, yeah. Okay. Maybe we can see here that I mean it thought for quite a long while here.

14:40 I mean that result right there was $6.36. This may be a shorter video. But the final result, unless I decide to do one more thing, I may. But so far, I'm kind

14:52 of having the feeling where I don't want to spend any more money on testing this model because we're just using it directly through Open Router. So, we're paying for it. Let's just take a look at

15:03 our watch website. This is the one where it also needs to have the special edition dubbed the Bjan and an exploded view of the watch. Okay, again, we're noticing that common theme of geometric

15:13 issues. So, there are elements here that are decent. Like the actual case of the watch, but it's just got so much issue in getting things correct. Like the prongs that should be coming out of the

15:24 crown here are just on the wrong plane. And this has actually like one of the better latches or strap pieces there that I've seen before. But it's just really like everything's kind of

15:34 crooked. This model seriously has some like deep geometric or spatial issues and that's very much let these results down. I'd say overall actually as a front end aside from the watch model and

15:46 things like that, this is not bad. The renders here would have looked good were this actually properly done in terms of like geometry. All right, let's take a look at our

15:57 exploded view. See, like that's not bad. However, I'd like to be able to move it around a bit. Like the face it drew is not bad at all. Balanced. Okay. And these are just

16:12 presets for specific portions of that exploded view. Let's go and see the different versions we have. Very cool. Okay. And the Bjan right there is pretty cheap considering what I would expect

16:24 that to be. And we see one of the big issues here is just the sapphire glass being crooked. But then on the back here, we see like some nice elements of it. So

16:34 definitely a preview model, how one is born. And overall, the front end here is really not bad. I think the final thing we're going to do just as a simpler end to this that hopefully will have a

16:46 decent chance of success is give it a test to do a pixel perfect playable replica of PvP at the Grand Exchange in Runescape 2007, and we'll see what we get. All right, let's take a look at our

16:57 GDO OSRS clone. I have to say this is actually not badly done. Reason being, I didn't expect this to use GDAU. Oh, okay. Normally, they use 3JS for this. I don't really specifically tell them you

17:13 have to use this one language here. So, the good thing is that it used some freedom. This is troubled because unfortunately it is

17:26 glitching out and failing at some point. Sit no prey. Cool. So, it's also put in lore. It definitely has pertinent knowledge about pking in old school Runescape. I think this is not bad as

17:38 just like a little 2D game like this. However, I think the big issue is the fact that it like these are names that some folks may be familiar with if they do watch

17:49 content related to Runescape 2007. Who would do that? So, it's cool. The big issue is definitely the fact that like there's no inventory, there's no real UI, there is

18:00 some combat logic, and it keeps failing and crashing. So overall, not really a successful result, but just a different demonstration of the capability. So the final thing we're going to do is just

18:13 take a look at how much this actually costs throughout every single test that we ran here. We saw everything that was done with this model. And the total cost for this was a little over $22. So

18:24 definitely kind of expensive, especially when considering the quality of the results we received. I have to say overall I am fairly disappointed. I understand this is still a preview

18:33 model, but it is such a significant leap in size over step 3.7 Flash. And I was far more impressed, if I do correctly recall, with the results I saw with the 3.7 Flash model as opposed to this. And

18:46 the competition that this is going to face is pretty severe. Even if we discount models that are closed source and proprietary, so take Haiku and Luna off the table there, this still has to

18:56 compete with Deepseek V4.1 Flash, which is around the same size. And I don't really see why I would use this over DeepSeek V4.1 Flash. The cool thing is it is soon going to be open weight. So

19:09 there may be some use cases for it as well. And perhaps it excels on some niche things. But in our suite of tests, I found that unfortunately the performance that I saw here was kind of

19:20 a let down compared to competing models like DeepSeek or even GLM 5.3 Flash. I think would be probably a better option than what was seen here currently. So overall, this is definitely going to be

19:32 a shorter video than normal, but I did want to test this seeing as it is currently available, I think for free in some like open code or something. So folks may be interested in using this

19:41 and I wanted to do another model test cuz it's been a little bit. So overall, that is going to conclude our first look and test of step five preview from Step Fun. Um, if you have any questions,

19:54 please feel free to leave them in the comments.