Cognition SWE-2 First Test – Is THIS a Better Kimi K3?

0:00 Hey, welcome to the channel. It's Bian here with another model test. Today we're going to be looking at SWE2 by Cognition. And this is a really interesting model that a few folks had

0:14 mentioned wanting to see tested and for darn good reason. This is basically a Kimmy K3 model that has been trained additionally by the cognition team to specifically excel and improve at

0:25 agentic encoding work. Now Kimmy K3 is a 2.8 a trillion parameter model that is already probably a frontier state-of-the-art openweight model. So seeing this capability increase that's

0:36 actually shown by this kind of neat chart right here where they showcase its performance at the beginning of training and then at the end of training on different reasoning effort modes, we can

0:45 see this does hypothetically represent a pretty big jump in its capability. So before we get into it, do feel free to subscribe. We're like one or 200 away from 75K and then on to 100 hopefully

0:56 not one. So let's get into it. So, let's start out by taking a look at some interesting things about this model. This announcement post does not have a whole lot in like pretty demo like

1:05 here's a sick game it made, which is exactly why I'm here. No, but they do have a lot of like uh technicals about training it and things like that. However, something interesting right

1:14 here is this deepsw SWE score chart. Now, DBSwe has been shown very recently to perhaps not be ground truth in judging capability, but this does show an interesting story where this is

1:24 essentially matching GPT6 Astra on medium. Are we to believe this benchmark and for a significantly cheaper price? So, this specific score was achieved with $4.38.

1:35 This one right here with $1.34. Now, I do have some problems with the cost here because the only way to use this model is just to use it from within their app or like through their subscription.

1:46 There is no like I'm going to use it directly from the API and it's a $134 per million out or something like that. So, keep in mind we don't have cost metrics for it in the traditional way

1:56 that we do for all of these other models basically through the API. However, this could still just show that hey, maybe I'll get to use this a bunch more on a $20 plan as compared to something like

2:06 this. And again, this is post-trained from Kimmy K3. So, it's pretty interesting to see that an openweight performant model was made more performant. As we can see right here,

2:15 the benchmarks versus the model that it was used to be created from. I don't know why I formed it that way. These are some pretty decent jumps in capability with some additional training. So, with

2:26 that, there's not really a whole lot else here to discuss, at least pertaining to our specific video. So with that, let's just now hop into Devon right here where I have pasted in the

2:36 specific browser OS test v2.7. It is selected as s2 on max. And you can see there are some selectable reasoning efforts, medium, high, or max. We'll probably keep it on max for all of our

2:47 runs here. And apparently there's this promo until 109 2026 and it has 262k context, which hypothetically it's free, but I had to sign up for the $20 a month plan to access it. So, I don't know.

3:01 Make of that what you will, but I just figured it would be fun to test this. So, I did. With that, let's get started with our browser OS test. Right here, I have the only thing I've done with this

3:10 is just ensure that it is set up to work with the Blender and GDO MCPs on this system cuz in some of our later tests, we'll want to use those. All right, we have received our result. Let me just

3:20 see real quick how many lines this is. Okay, only around,50. That's actually less than I expected. I don't remember what the original K31 was, but we'll see how similar or

3:30 different this looks. Okay, not bad. We definitely have like a GPT5.x aesthetic to this UI. Let's start out just by checking some of the simpler things. Good. Resize works. Now, the big

3:42 one. Yeah. All right. Good. There is a right click. We do have the date and time in the bottom right. And I would presume this is probably the special feature. Okay. Nova Assistant. So, we'll

3:52 definitely keep that for last. Let's check out our start menu. And we do also have some form of like synthesizer app, which always makes me excited. Oo, my paint. You can paint something and

4:06 then send it to the wallpaper. I kind of like that. Starfield is pretty. There was even a shooting star there. I kind of like neon waves, though. All right, let's start out with help and about,

4:16 which I believe it opened with. We already looked at settings. I'm just going to start with the GTA game because I'm most curious about this one. Okay, it's

4:27 it's you know. Oh, good. There is a city common area. Z fighting on the ground of there. The grass was coming through the pavement.

4:38 Can we get out of the car? Oh, apparently we can. [laughter] Okay, this is H. I would have probably expected a bit better for a K3 sized model, truth be

4:50 told. All right, let's look at our other game, which is Void Strike. Wow, this one's actually better than the GTA game. It's cool with the specific background

5:05 that's selected. It's very meta to this game. All right, overall, that wasn't bad. I guess we'll be going through this one a little quicker than expected. Let's try our terminal.

5:16 Nice. Is that a Q? Yeah. Yeah, that would be a Q. All right. Hack. Initiating hack. Uhoh.

5:30 Wow. It's probably in bad taste given all the current things going on with AI now. All right. acceptable mailbox prize at totallylegit.biz.

5:41 You won 1 million. Good. Sometimes when they put in fake spam emails in here, it's interesting. Additionally, there are just meta references to specific apps in this browser OS, which a lot of

5:51 them do. Can we delete things as well? Good. And it does go into the trash. What if we deleted from the Oh, okay. Paint. Now, do recall that we can actually set this as our wallpaper. So,

6:03 these are kind of tough to pick from. Spot on. Then finally we have synth FM. Oh, no way. [music] That one kind of slapped. And then we

6:32 have chillhop. Oh yeah, that's nice. Hey, welcome to the channel. It's Bian here with another model test. All right. All right. It said never the same twice.

6:48 Let's just Yeah, maybe. All right. Overall, not bad. And then the final thing, of course, was, "Hey, I'm Nova. I run this place. Type help to

6:57 see what I can do." All right. Overall, not bad. Simple, but you know. Oh, and we had cascade windows, which will just snap them if a couple of them are open. Yeah, not bad.

7:13 Next up, we're going to be trying the self-contained skateboard game. This is using C++, and this is the one where it needs to be an early 2000's New York City block map. I think this should be a

7:23 pretty good display of its capabilities, as I believe the original K3 did pretty good on this. However, don't quote me on that cuz it might have been a different map. All right, we've received our

7:32 single file skate game. Truthfully, it didn't take as long as I had feared it might. And again, that was on maximum uh reasoning effort. So, I was not in the correct directory. Now, we can take a

7:44 look at it. So, that was actually relatively quick. Okay. Now, oh, all right. We just maybe won't full screen this. It's quite simple. However, let's first look at it before passing any

7:58 judgment. You know, this actually isn't half bad. [laughter] All right, let's try this once more. The board moves independently of the

8:12 player. Overall, it's pretty low fidelity and simple. However, that doesn't mean it's bad. It just means it's graphically simple. I'm not noticing anything that is horrifically

8:22 out of sort here, except for the issue that happened when we were grinding on that rail. I mean, there's moving taxi cabs. There is steam coming out of the manholes. I don't see a fire hydrant

8:33 shooting water out yet, which is pretty commonly included with these specific tests. Interesting. And we even have some nice water. The buildings in the horizon. This is

8:45 actually, again, it's really simple, but that doesn't mean it's bad. Slammed the wall. Can we interact with the pedestrians? Nice. We can. Oh, we fell off the board.

8:58 All right, I guess that's fair. This is actually the more I see it, the better I like it. Now, truthfully, I think that this could use a graphical fidelity tip. Try typing mega plan to plan deeply

9:10 before building. I was not aware of that. All right, I'll keep that mega plan on right here and just basically say this is a really good base, but it's very

9:19 simple. It could use a lot of graphical fidelity improvement and we'll see what we get. The graphical overhaul took quite a bit of time. However, there is also now audio. So, let's just take a

9:29 look at this. Okay. You know, it's interesting because it did overall improve the graphical fidelity of this. Regrettably, I unless it just made a new build, which is

9:41 possible. I don't have the earlier version. However, there's definitely a very very prominent difference just overall in the textures and things like that. However, I find it interesting

9:50 that it kept everything pretty much exactly the same. So, this was truly just like a new coat of paint on it, so to speak. However, interesting and while this may not be as impressive as like

10:03 something from Astra or Fable, it's still actually a fairly competent result when considering that. All right, let's just hit a pedestrian and then we'll move on.

10:15 That looked really quite painful. Kind of realistic. All right, let's just Nice. Not bad. Oh, except for Okay, except for that.

10:28 Next up, we're going to be giving this the 1980s 3D wrestling game prompt. However, it is being told to use GDAU and Blender to create this game. I am not starting it out in any form of like

10:39 plan mode or the mega plan because honestly, I think it's a better use of that allocation of time to just look at what it makes as a first result and then if there's anything egregious, we can

10:48 just tell it then to fix it in a follow-up prompt. All right, we've received our wrestling result using GDAU and Blender for the assets. Okay, music on. I hear it. It's just Cognition

10:59 Sports presents. Okay, triple threat wrestling. Let's just enter the ring. Okay, we can pick one of these. Normally, I'd like to see a little more. Actually, I'm going to say this is one

11:11 of the more playable ones that I've seen before. All right. I'd like to have a little more sound. It's The sound is bad. I'm gonna be completely honest about.

11:21 However, um they seem to be stuck in like some form of [laughter] Okay, the atomic slam. It's uh I just saw like sweat particles flying

11:37 off one of them or something. I'm trying to just spam click the keys that are like as shown in the moves down there. Oh, okay. The warlock won. I'm going to

11:52 turn the music off. Enter the ring. All right, let's try one more time. Oh, nice. We knocked both of them out with the lightning jab,

12:06 the starship elbow. Oh, it was cool. So, it actually did do some basic form of rigging for this as when I used that like elbow um move, the elbows actually kind of bent. So,

12:20 I will say it actually overall did an okay job creating these characters and then rigging and animating them. I guess in terms of some of the movement, we're still in this, so that's cool.

12:31 Oh, we have the chair. Again, I'm basically just like spam clicking things, but it doesn't seem like we're doing any damage to Macho Marco,

12:45 who is who we're playing right now. And we lost. Honestly, it wasn't bad. It wasn't as stunning as I would have hoped it to be just based off of, you know, what I expected. But overall, it was a

13:00 very respectable result. And some of the actual movement of the like arms and things like that was not horrible. It was a respectable result. Next up, we're going to be doing the robotic arm test

13:10 where it gets a camera, a view of a robotic arm, and then a task to pick up a Hot Wheels car and move it to the other side. It's also given some information about the arm, such as the

13:20 instruction manual for it and like what USB device it is on the system. Overall, this was unfortunately one of the more disappointing results I've had with this test, though we have only run it a few

13:30 times. Basically, it was doing a lot of destructive actions where the arm was moving, trying to lift itself off the table, the edge of the arm was hitting the base of the arm, and then ultimately

13:40 the car got moved to a position that almost put it like directly underneath the arm, which was a bit odd. And unfortunately, it just wasn't quite getting it. So, basically, we can see

13:50 right here, it's done a lot of work where it essentially figured out, okay, I'm going to move the axes initially, which they all generally do, just to figure out what servo is which, and it

13:59 identifies them that way. And then it takes photos and looks at them and then runs a script to make a movement with the arm depending on what it sees in the photo. I think I don't specifically know

14:09 what went wrong right here, but I mean, let's take a look at one of these photos. Okay, so that looks much more bullish than it was in reality. I just want to point that out real quick. But

14:19 basically, that's what it was doing is it was just taking images right there, looking at them, and then deciding to make specific moves based on that. Something I noticed that happened and I

14:27 believe this happened with Deep Seek V4.1 Flash as well. It assumed that this was an open gripper position and then that that is a closed gripper position which was inverted. I'm almost wondering

14:40 is there something wrong in the instructions I've given them as some models have figured it out themselves? But that may be just going on their visual basis against the instructions.

14:49 So, I'm going to need to look into that. But it's just odd now that two models I've noticed have mistakenly identified open versus closing grippers. And then basically finally we ended up in a point

15:00 where it's hard to tell here, but the arm was so far away to the side of the car. And this is clearly lifted off of the plate. And then if we are to just compare this with where we had started

15:09 when we begun this test. So we can see right here the car moved a lot and then this thing was basically lifting up off the table and just unfortunately didn't get it. Next up, we're going to do the

15:19 Subway FPS game. I am for this one going to begin it from this mega plan mode. It is, of course, also on maximum reasoning effort because this will give it the best chance to just out the gate produce

15:29 something very visually compelling and nicely playable. So, we'll see what we get. It will initially create this plan. We may have some follow-up questions to answer depending on what it wants to do,

15:39 and then we'll let it go. After about, I think almost an hour and a half, if not a little more, we have our result right here. This worked for an incredibly long time. I don't know how much of that can

15:48 be attributed to having begun this just with that planning mode. All right, we'll see what we get. And keep in mind, this is an updated version to this where the waves come like they get delivered

16:00 by train. Okay, it's for some reason muted. There we go. All right. Oh no. Okay. See, like that's awesome, but that's an issue. That's not supposed

16:16 to happen. The enemies are Oh, yeah. We can actually go down in the track. I believe that if you get hit by the train, it will kill you. So, keep that in mind as well. One of the biggest

16:27 issues, which is really actually kind of cool, is just the way that the ammo and the tracers stay frozen there. We also have a shotgun, and the blast for that is actually like

16:39 how a real life one would be. Now, one of the big issues I'm noticing here, aside from the weapon tracers, is just that unfortunately the weapon models are kind of just like low rent. Okay,

16:52 nice. Nice. That is um looked better from further away. All right. And the train just backed out. That's odd. I don't think we're Oh, I don't

17:09 think we're going to make it past this. So, we do have bullet holes in the environments. And unfortunately, I can tell you for certain that this is better than the K3 result alone. I think

17:25 I may have to verify that, but I think it's overall just a better like more detailed scene. So apparently basically like you can get hit by the train.

17:41 Yeah. So that was a interesting thing. I have to say it's [snorts] it's again it's competent, but there's been so many models lately that have just been so incredibly outside of the league of

17:53 anything else that it's tougher to judge things like this now. I feel like if I had seen this 6 months ago, it would have been like, "Oh my goodness, this is the best one." So, it's tough to judge,

18:03 but I did just want to test it on something that's a little more traditional that we have a lot of prior reference to compare against. Next up, I'm going to be giving this the task of

18:11 creating the 3D print files for a specific engine. In this case, an inline 6 RB26 engine. This is a test that I specifically want to do with this model because the original Kimmy K3 when it

18:23 came out produced this, a bunch of different pieces designed to fit inside of it, a much bigger RC motor. Reason being that I still have this, it's actually still to this date one of the

18:35 best results I'd received for this specific test. Now, the specific engine we're having it make right here is different than what I'm holding in my hand. I did change it up just to like

18:44 prevent benchmaxing or anything of the sort. However, this was still at the time, especially when this was seen, it was basically jaw-dropping like wow, for an openweight model to have produced

18:55 this quality was really quite striking. And still, this looks darn good. I have it still here for a reason. All right, after just under an hour, we have our 3D model created. I'm very eager to take a

19:05 look at this. Okay, we do have a Blender file as well. I had totally forgotten it had created this in Blender. So, let's just take a look at the parts individually. For some reason, the STLs

19:16 aren't actually opening in the printer slicer program. So, we'll just uh look at the Blender scene. So, here's our result. And we're just going to have to look at this just from within Blender

19:27 right here. Overall, it's not bad. However, there are some things that I'm truthfully not thrilled about. Basically, the turbochargers are actually some of the better ones I've

19:39 seen when doing this test. Make of that what you will. The intake manifold is not necessarily oriented correctly. However, I'm going to say the actual level of detail on the front of this

19:50 engine, like what we see right there and the pulleys and things, is not actually half bad. Additionally, it did properly orient this so that the motor shaft would come out the back of the engine as

20:00 a more realistic thing were this to be placed in an RC car. That's what you'd want for the realism factor. Let's take a look at some of the individual pieces right here. Okay, so we do have a kind

20:11 of floating bit, but just assessing this, I don't see anything that would really prevent these from being 3D printed, which is nice to see. These would print individually. They would

20:21 more or less be fine without support except for that overhang right there which we'll let go. And then let's look at some of these individually. The top that would actually print nicely in one

20:30 piece. It did take care to put like overhangs at a degree level that wouldn't necessitate supports. The only problem is that little piece right here would kind of not be good. Additionally

20:42 to that, let's take a look. Okay, strap. So that was actually just a separate piece. We have the manifold right here. that would print nicely. And then finally, the bottom. So, let's take a

20:52 look at the cavity that the motor would go into. That's actually that's acceptable. So, overall, even though we didn't really get to look at it in a printer slicer

21:02 program, this is not bad. It does have alignment pegs to pop things on. Only a few issues in terms of printability. However, overall, this was a decent enough design. And we have the twin

21:11 turbo right there. acceptable and something I wanted to do, especially considering that K3 had created such a fantastic result in one of my earlier versions of this test. And this also did

21:21 use Blender. So, it may have I don't know what to make of this, but I wanted to do it. [laughter] Next up, I want to give this the watch website test because this is always a good view of the front

21:31 end where it creates a 3D model of the watch, a Keyshot style render of it, and then cinematic camera angles. Also, the exploded view. Now, I have also included just an additional thing to maybe spice

21:42 this up a bit. I've given it this specific photo right here and said there must also be a special watch called the Behen, which is which with the attached photo placed nicely on the dial. This is

21:52 a oneofund special edition and very high-end. So, we'll see what we get. I'm very eager to see the result for that specific prompt. For the Bejian watch variant, I need to embed the provided

22:02 image as a B 64. All right, we have our result. So, let's just look at it. I'm very eager to see the Bjan edition. Okay. Time distilled. I believe K3 used that exact same lingo

22:15 in the page, which we'll just maybe ignore that for now. Additionally, I'm noticing on first glance, this is good. The crown looks good. The materials look nice. The big issue is the watch face is

22:27 oriented 90 degrees clockwise when it should be like the other way. You know what I mean? Hopefully. Interesting. And it's kind of hard to see above. So maybe we'll scroll down. I did see some

22:41 nice blue reflective properties of one of those hands. That was good. All right, let's see. An obsession with the unseen. And it has the traditional black and gold and white

22:52 aesthetic. The Riviera collection. Okay, cool. We have ooh 219 or 124. Both looking good. They are spinning nicely. And the second hand is actually moving. Uh, now we can see there are some

23:05 misconfigurations here in the numeral markers, which does happen. All right. Now, the big thing I'm interested in, which we can see here in the top right as well. I'm just going to scroll down

23:14 and we'll see if it shows up. We also do have the exploded view. That's nicely done, I would say. Aside from the misorientation here, can we move this

23:27 around? Unfortunately, we can't. So, it's hard to see if there's any real movement implemented in here, but we do have like some screws and things like that. At least pieces of it. So, that's

23:35 good. Now, I'm too eager. Let's just All right, here we go. The Bejian, the [laughter] the man, the myth. $95,000. The man, the myth, the dial.

23:48 Unfortunately, what's letting it down again is the orientation being wrong. But if we just take a moment and look at this, I'm quite pleased with this to the point where I may have to figure out how

23:57 to get like a custom dial or a face printed and then put it in a watch. So, the second hand is moving nicely on this one. And again, the numeral markers are messed up, but that is okay because the

24:09 whole point of this was just, you know, [laughter] request allocation. You can't even buy it, which is good. Once a year, the Mult releases a watch that needs no

24:21 introduction. The Bjan pairs a solid gold 18K case with hand lacquered portrait dial. The chairman himself sealed beneath box sapphire and flanked by a fluted gold bezel. Each piece is

24:33 assembled by a single master watch maker and accompanied by a signed certificate allocation by application only. Very well done. Well, at least that overall. Again, this was acceptable. It was good

24:44 aside from some issues, but the models that have been coming out lately have really had like distortion of perception on results now. It's much harder to judge things that don't wow in the way

24:54 that things like Astra or Fable do. So keep that in mind as well. So that's going to conclude our first look and test of SWE2. I want to quickly just show as well that the usage has not

25:04 really moved at all being that this model when selected is shown to be free until October 9th. So that is why no usage was really used up. though, I'm pretty sure I did need to sign up and

25:15 pay $20 a month to actually be able to select this model. It's possible I made a mistake there, but anyway, I just signed up so I could test this cuz I figured why not. Really, overall, I have

25:25 to say it was performant, but again, it's become so difficult to judge some of these outputs now based off of what's previously been seen. And this is benchmarking pretty high from what we

25:35 saw in the announcement post where the results should be a stones throw from things like Opus 5 or Fable or GPT56 Soul. For a quick results overview, the browser OS was fully acceptable. Nothing

25:48 really special here, but it doesn't really give a huge opportunity for the model to showcase anything jaw-dropping. However, the music app was always a welcome surprise there. The skate game

25:59 was interesting because the first time it was pretty simple. It remained simple when we told it to overhaul the graphics. However, it did definitely add a level of polish and visual fidelity to

26:09 these graphics, even if they weren't necessarily like changed at all from their simple style. And aside from that, so that was really like the biggest issue we noticed was the grinding logic

26:19 there. But this was actually it was cool. And the more I looked at it, the more I liked it. It was nice to be able to crash into pedestrians. We had a view of the water and then some buildings in

26:28 the horizon. So that right there is pretty cool. Can we actually jump into the No. But this was a definite improvement over the skate game result that I'd seen from Kimmy K3 by itself.

26:38 Even though it was set in a different map, it really showcased a leap in this specific test. Then had our GDAU and Blender game, which was actually surprisingly decent. It wasn't very

26:48 visually exciting. However, playability on this one and the movement of these individually like modeled characters. Now, the more I look at this, again, kind of like with the skate game, this I

26:59 think actually may be one of our more impressive results because these are difficult to get right, most of them don't end up really being playable at all. And this, while it had its quirks

27:08 and features and things like that, it was more playable than a lot of the ones I've seen before. So, this was actually pretty nice overall compared to what normally gets generated. Then, our

27:19 Subway game was not bad. It just was not at the level of what I'd hoped to see, which would be like fable level or something of the sort. However, I'm going to say that it did a nice job

27:28 implementing some of the things we wanted, like the train showing up and things like that. The unfortunate glitch here was just the ammunition tracers getting stuck in midair, which really

27:38 looked absolutely fantastic. However, just kind of like it's a glitch. It's not intended for that to happen. But there were bullet holes in the environment, and the overall aesthetic

27:47 here was very grimy. This was again run on that mega plan. So we did that first and then had it generate the game once the plan was complete. This took I believe like an hour and a half from

27:56 start to finish as well. Starting with the initiation of that plan and then building the game. Then we had our 3D engine model which was really not bad at all. It did make it in Blender and

28:05 finally we had the watch website which we don't need to look back at. However, this was um very enjoyable for me personally and something I may include sparsely in this prompt as well just to

28:15 spice things up. So, that's going to conclude our first look and test of SWE2. If you have any questions, please feel free to leave them in the comments. And thanks for watching.