Qwen 3 8 Flash Next VISION Local Ai Test

0:00 We're checking out the visual reasoning on the latest Quinn 3.8 Flash Next, which is a much larger model. We had some amazing results when we checked this out with Quinn 3.827B

0:11 at full precision. Today, we're going to be checking this out at a quant 4 size, but this has landed as well as some other performance improvements. So, if you are wanting to follow along and run

0:21 this, you do need to pull from source and build fresh because that will bring in your vision support and also some other performance improvements. And when I say performance improvements, take a

0:31 quick look here on the top. You can see this is the benchmark that we ran for the original. This is the Unsloth branch. And now that it has been merged and some additional updates made, you

0:44 can see that there is a substantial TG improvement going from 43.99 all the way up to 62.42. So definitely definitely worth it just for that alone. But of course also the

0:57 support for vision brought in here. So really quick going through what my config looks like that we're going to be running here. You'll note that my mm prod is now specified here as the BF-16.

1:08 And if we scroll down just a little bit more, you can see that I've changed it back to load type of mm map. I have actually 262 144. So the full context window size do have parallel set to one

1:19 and I've got my batching at 2048 and you batch set to 512. This probably is a little bit tuned for visual reasoning specifically and I definitely wanted to make sure that I had XH high set. Also,

1:32 the rest of this is kind of just what you're supposed to if you're going to have reasoning enabled down there. Image min and max tokens. So, giving it a very big amount of budget here, but does look

1:43 like everything's actually been working pretty good. So, before we get up and running here, this is running off of again the Quad 3090s. I do have the 4090 also on this build, but it is just the

1:54 quad 3090s that are running at the moment. You can see that over here reflected as we have our llama C++ running and we've also got our quad3090s in in NVTOP. This is running on our

2:06 Proxmox host, all of that good stuff. You can find out how to set up and run by checking out the channel history. Tons of great information there. And huge hat tip to all of our channel

2:14 members. Be sure to hit like, subscribe, and if you're new to the channel, welcome. Definitely this is a still bigger model than a 27B. However, it is the flash next so it may not be quite

2:26 the same fidelity. I have questions about this. I've checked out before in the past some of the quants as far as vision and not got the same result as the BF or you know full precision that

2:38 you can get with running VLM. So that's one thing to keep in consideration here. We'll check this out today. And like I mentioned, we saw amazing results with the Quinn 3.827B

2:49 at full precision with VLM. So, the first one we're going to test out is this one got everybody kind of like, whoa. And this was just picture of a camel in the Hill Country. Check that

3:00 out. Camel Hill Country. One of my neighbors got that going on. Now, being able to tell that it's in the Hill Country of Texas though versus somewhere else, that had people like, whoa, it

3:08 just got that. And it got that like right off the bat the first time. So, I'm just going to ask it what it sees and to take a guess at the location. We'll see if it can get it. The Quinn

3:16 3.827B nailed it. Just absolutely nailed it. Dramadary, yes. One hump, yes. Shrubs, yes. Most likely Texas, yes. Hill Country, South Texas. So, South Texas,

3:29 you could see this also very frequently. Hill Country, there's actually a lot of these around. It's not a wildlife park, but definitely there are wildlife parks that do have camels and other stuff

3:40 there. So, it did get where that's located at. However, it, you know, didn't firmly call it out as the Hill Country itself, and you can see it's kind of making a face there. So, I'm

3:50 going to make that same face and say good, but not quite as good and on point as we saw with the Quinn 27B. That was insanely on point. Next, we got a weird little bug. Um, what is this thing?

4:05 And this is, I believe, called a false robber fly. And these dudes are all over the place. And they're kind of like predatory. I believe they go after other bugs and insects. They kind of look a

4:18 little bit like a miniature dragonfly, but they're not dragon flies. Okay, it did get it. It is a rubberfly. That's correct. Definitely got that. And move on to the next one. And this one

4:32 here, let's just So, some people pointed out with both this one when I asked it the other time in the comments, really good question. They were like, did it have XF data or anything? It does not

4:43 have any. There is no XF data on these. The ones that I'm going to ask location about. So, these are quite old photos. This one here, properties on this one, no XF data

4:57 existed on that. as far as the GPS or location as well. So, this next one, we're going to just ask it where this is and we'll see if it can get that location. This is actually on the 360

5:12 Westside Trail looking towards Austin. That's uh Lake Austin that you've got right there. Golf course, a lot of the kind of natural limestone bluffs that you have in the Hill Country area right

5:24 there beside it. See if I can get it. Running at about 52 tokens a second right now. Not Lake Travis, importantly. Oh, it's going to try to be vague. This

5:37 is not Lake Travis, and it is kind of the same visual appearance possibly, but it does not appear. Yeah. Okay. So, it's got Austin Skyline there, which Austin Skyline probably from this location is

5:51 not quite 20 miles. If it was Lake Travis, yeah, it probably would have been, but this is Lake Austin. So, gonna give that just a fail. Pull up a new one here and gonna ask it. I mean, I think

6:02 it looks apparent what it is, but it might not be able to pick this out. And we'll just see whether or not it can pick out what this I'm just getting hungry now. This is what I'm going to be

6:10 doing later on today. I can see it being confused because it's kind of a black surface on the griddle and you can see it really utilizing the GPUs here for this. This looks like a

6:24 closeup overhead shot of boogers being cooked on a flat top griddle. Ding ding ding. Correct. Simmer slick with simmering oil. Brown bits from the meat. Yes, it does have that delicious

6:35 delicious brown bits from the meat. On it said several beef patties, each topped with a layer of pale melted cheese. looks like white cheese such as Swiss and it is. And some toppings on

6:45 the top and right burgers are piled with sauteed yellow bell pepper strips with a touch of some red peppers and some caramelized onions on the right one. So yeah, actually caramelized onions were

6:57 on both of those, I do believe, but I think there were caramelized onions on all of them. I don't know if it'll be able to distinguish that. The caramelized onions, of course, looking

7:07 pretty close to everything else. The left and front burgers are covered in brown sauteed sliced sauteed mushrooms. And yeah, that's it's not 100% accurate, but I think it's pretty close. I mean,

7:21 this is quantized vision. So, this is one of those trade-offs that you might be want to be aware of if you're running this for specifically vision purposes is there's probably better options out

7:32 there. I have seen this before. Like I mentioned, when you start quantizing vision down and asking it to do things like read an LCD, the accuracy is substantially different. Also, this one,

7:43 no XF data on that. This is a old skyline. So, this is what Austin looked like uh 2011 2012. We'll see if it can get that this is Austin because it may not, unless

7:54 it's been trained on like kind of historical photos or unless it knows per building details, it may not be able to get this. Seeing if it knows the time frame would

8:07 be interesting. Oh, the skyline looks so so much different now. They have these really fun pedal bikes you can go out though on and you can take your own beer out there or wine or whatever and drink.

8:18 So, it's kind of fun. I think it's going to get it though. Looks like it's definitely going to get it. But, will it be able to get the time? I think that's actually a little bit

8:27 hard. Most people that are in Austin right now don't remember when Austin had that few skyscrapers. When I first moved here, it had substantially lower than even that. It's like four. Okay. So,

8:39 it's getting the Austinonian 2008 timeline. So, yeah, the Aonian kind of the big one here in this picture. No longer the big one. Everything's like wallto-wall now. So, I guess you have a

8:51 view of your neighbors windows probably in most of these mid 2000s to early 2010s. So, up to 2012. And yep, I believe this photo was Let's take a look at it.

9:08 2012, June 7th. So, yep, got the time on that also really well. And I think that it went through a process of elimination. So, really good on that one to get that one. And next up, I don't

9:21 actually know what this bug is, but we're going to see if it can tell me. Uh, what is this thing? Ramping up those fans on device zero.

9:31 You can see that is 100% utilization. Truly pegged and 78 centigrade fan running at 100. 335 out of 350 watts being used. I did not cap it. Don't have to cap it.

9:45 technically got enough uh power for it on this system, but should probably power cap it. Actually, probably you don't really usually see that big of an impact on image

9:56 processing and video generation, stuff like that. Yeah, you probably want to run it higher, but I would imagine going down to like 325, going down to 300, probably not that big of an impact for

10:07 just processing images. I lied. I know what it is. It's an assassin bug. However, I've learned something here. broad-headed bugs and this family ali. Did I murder that? I'm

10:20 sure people are going to tell me in the comments I murdered that. White is a bug and not a wasp or an ant. We have so many wasps and ants out here, but it is definitely not a wasp or an ant.

10:27 Noticeably wide flattened head, does it? I don't know. It kind of has a very narrow little head like a It's not big. And then it's got this little like thing that sticks out of it. Yeah, it's asking

10:38 me for more pictures and stuff like that. So, I would say that I would give that like a 75% because it is, to my knowledge, called an assassin bug. The other stuff there, it seemed a little

10:48 wishy-washy on whether it was, whether it wasn't. So, I give that a 75% pass. Let's see. So, this here is how it starts before it goes onto the Blackstone. So, yummy, yummy, yummy. I

11:01 guess I'm going to ask it to name all of the ingredients that it sees here. And there are quite a few. So, we've got bell peppers. We've got some red ones. We've got some yellow ones. Mushrooms.

11:13 An avocado. More mushrooms diced up. Onions. Baby Swiss. Green beans. Notably, hamburgers are not present here. So, we'll see if it can get that or not. There is oil

11:27 back there, I believe, in oil. I don't know if it's going to get that or not. Hopefully, it does not consider the Dawn Platinum to be a food type or a food substance, but we'll find out what it

11:37 comes up with here. Check. Green beans, cheese, onions, mushroom, avocado, and bell peppers. Yes, it did get everything there. And I I think that is totally a pass. Next up, 3090. Before they became

11:49 3090s, they looked like this when I was cleaning them up. And you see how ooky and nasty that was before it became beautiful and pristine after being scrubbed with alcohol and repatted and

11:59 repasted. So I'm going to ask it actually to take a very highly educated guess at exactly which GPU this is. Note two eight pin power connectors up there. So maybe it should be able to get this.

12:14 I feel like it should be able to get this if it just comes back with this is a PCB from a GPU, which h you know, I'm going to guess that's what it's going to try to do is kind of give the wiggle

12:24 room. It seems like a lot of it's like wiggly, not like super precise. This is exactly it. And if I was running this at full precision, that's my real question is would it get there? Because the 27B

12:38 was definitely there. Like definitely there. All right, it got that it had two eight pins, so that should preclude a 1650 from this. The P2000 is not something that it

12:54 should have even considered there because P2000 looks completely different than that. In short, I didn't want you to be short. I asked you exactly. It's not giving me

13:07 what the card is exactly. It's telling me stuff like, you know, what I need to do and all this other stuff. Not what I asked for. Um, probably a pretty good indication that

13:16 it does not have a boardbyboard kind of knowledge of PCBs out there potentially, which I think that's pretty interesting. I think if I just showed this to several people I know, they would be able to say

13:31 this is that type of a GPU, like a 3090, because of the specific arrangement of everything on it. And moving on to the next one. Kill a watt on the wall. So we've got a kilowatt on the wall. Reads

13:45 8 watts. Pretty hard to read that offaxis. We're going to find out whether or not it can see and read the LCD. Oh, it got it wrong. That's actually 08.

14:00 But it's so close. Very close. But 08. It it was it could be nine. It's not nine. It was eight, but it could have been nine. And it could have

14:14 just been a little bit faded. But unfortunately, that one is a fail also. Close though. So, I'm not going to go through the whole garage DC and tell me everything because clearly the precision

14:25 is not there like it was with the 27B. Now, of course, I'm attributing at this moment to it being a Q4. I probably am going to need to go and test out the full precision one, which means it'll be

14:36 a very slow test because it'll have to run off the LLM on the safe tensors in split mode. And it will, like I said, be a very, very slow test, but it should be able to fit into the rig that I've got

14:48 here, I think, possibly. So, I'm going to have to give that one a run. But, I am going to ask it a fun one. And if you remember back, if you've been a fan of the channel and you've been watching for

14:57 a while, you remember this. And basically it was a question along the lines of is this a cat? And this was one of the gifts that we had generated that looked

15:11 a little bit more like an L or something to me. So we'll see whether or not it thinks this is a cat. Certainly everybody was split pretty much 50/50 on whether or not it was a cat. Hundreds

15:21 and hundreds of votes on it. But yes, it reads as a cat just drawn in a very minimal geometric line art style. All right. And uh so I'm gonna give it a uh that passes because really I don't know

15:32 what the right answer is. It's not clearly a cat. But you know what? It can see the uh the whiskers there and the big eyes and the ears even though the ears are not in the right. Maybe it's

15:43 looking over its back. Somebody said that and I was like, "Yeah, you know what? Maybe the cat's looking over its back at you." And cats can kind of do that with their head. They can just So

15:51 maybe that's it. Maybe that's what it's doing here. So, that is a good look at what the vision processing capabilities are with the Quinn 3.8 flash next. And I would say at Q4 for sure, I am not

16:08 wildly impressed. As a matter of fact, Quinn 3.827B at full precision smoked it. So, looking forward to seeing what you guys think about that. But definitely if you are

16:19 thinking about running one of these and using tooling for instance for getting information back and forth about user interfaces this is probably going to be a much

16:29 different story that this would be acceptable for doing that and like if you're setting up heresies agent and computer use so that you can maybe get some shots of your screen or something

16:38 like that that probably is going to work out pretty well and I think you'll be able to actually get that information that it would need to make iterative design changes to make things actually

16:48 visually flow on the screen better which is one of the key components of the front-end design coming out so good that we've seen with the agentic development versus like oneshot development or

16:58 zeroot development and stuff like that that we get kind of more a historical look at with things like an interface like this or open web UI. That's it. I would say me that's all I got. That's

17:10 all I got. I wish I had more for you. I wish it was a little bit more perky on this, but on Varnish Truth, that's what I think. And a huge hat tip to everybody who likes, subscribes, and of course all

17:20 of our channel members, all the buy me a coffees, and all the Patreons. You can learn more about that in the description below. And if you're interested in how we put together Budso here, this massive

17:30 rig, all those links are in the description below as well. And that build guide is coming out very soon. If you're looking to get going with a massive playlist of just AI non-stop

17:40 videos, check out the playlist we've got here. It goes on and on and on.