Running Qwen 3.8 Flash Next on a $1000 Local AI Build with Strata

0:00 This is insane. This is This is crazy. Local AI just got cheaper. Today, we're going to be checking out the Strata engine powering the Quinn 3.8 Flash Next model. We're going to be doing this on a

0:11 $1,000 rig. $1,000 rig. I think that's exciting. Let's get started. We've got two 3060 12 GB GPUs. We've also got 64 GB of DDR4 2133. For the processor, I have an E5 2696 V4. The Dell Precision

0:27 T5810 is about a $150 box. We've got a one TBTE SSD in here. Really, that's it. The rest of this is ready to go. They just came with everything you need, and they can support both GPUs at a full

0:40 PCIe Gen 3X16 wide. We're going to run the four different models that are available. So, we are just going to ladder this with our animated SVG of a cat walking on a fence. And you might

0:51 think that is a surprising choice in what we're going to use to evaluate quality, but I think you're going to see something pretty cool as we move through this. So, you can see right now these

1:01 two 3060s are cranking about 50 60 somewhere in there tokens per second. And if you haven't seen the Strata dashboard yet, there's not a lot that you can do to it. There's not like a

1:12 chat history. If you check out the monitor though, this is actually pretty cool. So, you can see the speed. It'll show you some information about your well at least one of your GPUs. It won't

1:21 show it about two GPUs. Shows you that we've got 50% of our GPU being utilized here. 22 cores, 44 threads of course on that GPUs a little bit hot. I definitely need to repad these 3060 12 GB GPUs.

1:34 They have been used and abused. The VRAM 24 GB available. So, it does also give us a little bit of an idea about the power load on the GPUs and that is clocking in around 200. I'd say the rest

1:46 of the system you would need to add somewhere around 100 to 120 watts. So, most likely around 350 or so is what we're running. You can see the context window that we've got down here is 256K.

1:57 Very impressive. And it'll show you our actual utilization of the system RAM here. Only 41.8 out of the 63 we have available. But we're going to look at the outputs. And this is a really good

2:08 way actually for you to see the difference in the quality that we're going to see. and the end one. If if I can just say if if you're going to jump anywhere, jump to that end one. You

2:17 shouldn't jump around. But you if you are going to jump anywhere, jump to that end one after you see the first one because it is insane. Don't see the first one and be like, "Ah, this is

2:25 crap." Go to that end one. And you can see the tokens per second that we're able to generate coming in over here. And it's about 63 or so. You can see that I've got my Hermes agent up here,

2:34 but that's actually running on VLM on the Quinn 3.827B, which is kind of a big brain. And I told it, hey, set up the machine over here. Did it perfectly. There's an agents read me in there, and

2:46 it goes through that, reads it, gets it up and running, tunes it for you, and does a great job of doing it, honestly. So, you don't even have to really spend that much time. So, we've got our first

2:56 animated SVG coming in here. There is also no like artifacts displayer. So, we'll be copying this. I can see that it didn't include HTML with it. So, we'll be copying this and putting it into a

3:07 file and taking a look at the cat. And I will say this one, I can tell you already, don't have high expectations for. It gets better and better, but the end one, wow, the end one's going to be

3:18 really cool. And let's give it a check out. And this is a pathetic and sad looking cat. Just barely barely able to walk across the fence there as it's got almost no ears. It's not a happy looking

3:30 cat. Everything about it is a little bit sad and the fence isn't even that good looking either. So that is your Q2. And this is again, of course, the reason I'm doing this, so that you can get a little

3:40 bit better idea about what you might expect here. You can see that came in at 15,275 tokens and 63.3 tokens per second. And next up, we're going to take a look at

3:51 the IQ2 XS. And again, all this is running back there on the T5810. And if we check out the monitor, you can see that this one is using 43.2 GB of

4:04 the system RAM. I was like, "Wow, this is 70,000 tokens deep and it hasn't really started outputting it." But here it is. Finally, we are getting our output. So, we'll be able to check out

4:14 the hopefully really good IQ2XS. This is of course one step up from the Q20 as far as the size of it, but really the amount of thinking that this did substantially more tokens and that

4:28 finished up at 85,844 tokens at 62 tokens a second on the TG side. Okay. So, it created an animation. So, definitely the fence is not moving. The

4:41 background houses are moving. It does look like it's got some issues here. So, not necessarily. Oh dear. Okay. Well, luckily I didn't move the mouse there because that would

4:55 have been a little bit tough. So, you can see where this was going, but it definitely had a lot of implementation issues as far as the quality of it. So, that was the IQ2XS.

5:07 Now, let's move up to the IQ3XXs, which should be yet again a little bit better even, and see that we've got the IQ3 XXS now. Toss in our super sophisticated prompt.

5:24 And it's off. All right. And it looks like we're there. We've got about 38,000 tokens. 37 815 exactly at 52.4 tokens a second. Well, this is actually pretty good. It's

5:39 a little bit crazy. Whatever this outline is that's going on here, we got some leaves falling. Definitely a very good fence, the moon, some birds flying, shooting star. I

5:53 mean, you can see the progression. This one at least makes sense and is actually functioning properly. The leg motion is a little bit crazy, but the tail is that's a big tail for a cat. I would say

6:04 overall though, this is and I think you would agree a lot better than the last one that we saw. So, final one that we're going to have is the IQ3S. And this is the big one. So, hopefully we

6:14 see an even better quality in that one. And we are on the IQ3S now. Coming over to the monitor, you can see that that's occupying about 53.4 GB. All right, let's take a look at what we

6:28 have here for our final result. It's not fast, but it still hopefully is good quality. Oh my gosh. Yeah, this is actually Okay, so there

6:41 there might be some anomalies down here kind of at the base, but the rest of this is actually really pretty darn good. And I mean, yeah, it also has a parallax

6:54 effect. Like it my moving the mouse around on it is making things change. If I click on it, it pauses. Uh let's check out the effects here. So noon. Okay, got our ginger cat going there.

7:08 And night. Seems like it turns into a gray cat a little bit, but still I guess that's a night cat. Show rig. Okay, let's check this out. Oh wow.

7:21 This is this is where we're at with 64 GB of RAM and two 3060 12 GB GPUs on a Dell T5810 from 2015 16. Uh so like this is just crazy good. This is just absolutely

7:40 crazy good. Hit space. It says to pause. Does that work? Yeah, it does work. Let's see if we can change the tempo of it. Oh, look at it sail now. Got some birds

7:53 in the background. And let's slow it down. Nice. This is insane. This is This is crazy. That's cool. I didn't actually ask it

8:11 for all of this stuff. It gave an explainer of the spec, how the viewbox is progressing. This is just not even close to what we've seen from so many other models out there when it comes to

8:23 the quality and also the capability in a small size. Like this is not an insane rig. This is like a,000,025 rig. Like this is this is insanely good. That is what is surprising me here. I

8:39 don't think usually I would have expected this. Wow. I've got to rethink a lot of things. I can assure you there's going to be a lot more videos on strata coming up here. Also, if you're

8:48 interested, make sure you hit like and subscribe because this is the kind of content this channel produces. Also, huge hat tip to everybody who is a channel member and I definitely think

8:56 you can check out more about that down below. Everybody who buys me a coffee and a Patreon, thank you. Also, if you're looking for more information on how to get up and running with this rig,

9:04 the entire build spec is down in the description and a link to it on the website. But wow, just this is crazy. Like should it be this like it shouldn't be

9:17 this good at this? And coin 4 isn't even out yet. So one of the real things that I am very excited about strata about is it should be applicable much of what we're seeing to quantization for Quinn 4

9:29 when it comes out because this is Quinn 3.8 flash next which was technology preview for Quinn 4. so applicable. Oh my gosh. Like you just don't you do not need a crazy big rig to do crazy stuff.

9:44 You do not need a like do you need 128 gigabytes of anything? No. Like 64 insane. Okay. Well, 64 and a couple of 3060s, but still this is insane. I look forward to reading your comments

9:58 down below. If you are looking for how to get up and running with rigs like you know this one and of course a lot of other things, check out our guide here where we've got a infinity of videos

10:07 that are all local AI related