Qwen is cooking customer AI..

0:00 Of all the Chinese labs that are all competing in the AI race, Alibaba Quen has been cooking recently specifically for models that run on consumer graphics cards. Labs like Moonshot, MiniAX, Z.AI,

0:11 and Deepc tend to focus on making models that are bigger than 70 billion parameters in size. Alibaba's approach with Quen has been this shotgun approach by covering a wide spectrum of models

0:22 across all sizes. And one that really stuck was models in this band right here where consumer grade graphics cards can run them. Hugging face made a recent report that shows this distribution of

0:32 model releases in 2026. As you can see, Alibaba covers a wide spectrum of models on all categories like under 1 billion parameters category to 1 to 8 billion parameters in size, 8 to 70 billion

0:44 parameters and over 70 billion parameters. And so far in 2026, Alibaba released a total of 51 models and variations already flooding the market with a wide spectrum of models. So while

0:57 other labs continue to push what's capable in the frontier of intelligence, what's worth noting here is this Quen 3.8 27 billion parameter model right in this band. And in the benchmark, it's

1:07 landing near GBT 5.6 Luna and Opus 4.6 level as you can see. And this completely changes the economics of intelligence upside down. The continual infusion of small and capable models

1:20 like Quen 3.827B A 27b totally eats away the aggregate demand that's currently met by data centers that are operated across the country by inference providers. And now we essentially have a

1:31 GPD 5.6 Luna or Opus 4.6 equivalent model at home without necessarily relying on data centers. In other words, not only did we see the gap between open and closed models closing in due to

1:43 models that compete in the frontier like Kim K3, GLM 5.3, DeepSc V4 Pro, which alone is crazy. But we also have a gap closing in between what consumers can run at home and what models are

1:57 typically only available in data centers. To give you a bit of perspective, Quen models already surpassed two billion downloads in 2026 so far, showing you just how fast the

2:08 adoption of AI is growing, not only in open models, but people who are trying to run open models locally at home. Another way to look at it is this AI stack right here. While the US has an

2:19 advantage at the chip and infrastructure layer over China, models like Quan 3.827B A27B is showing us that we still have a lot more to improve before chip and infrastructure layer becomes the

2:31 true bottleneck in terms of model innovation. Now, I think what's worth noting here is that just because Quin 3.827B is a small model doesn't by virtue make it a good model. There are

2:43 plenty of other models right here in the 30 billion parameters range from labs like Google, Nvidia, Mistl, and DeepSeek, but Quen 3.827B 8 27B is single-handedly a unique model that

2:55 punches way above its class. For example, Quen 3.827B currently ranks ninth place in Code Arena overall, while the similarly sized Gemma 431B scores 80th. So, out of the entire spectrum of

3:08 models here, this band right here is really dominated by Alibaba by a huge margin. Now, of course, given how goodw's models are getting, people are going as far as saying that they should

3:18 just now cancel cloud membership and just run AI at home instead. While I think it's still premature to say that across the board, it does bring to question where the trajectory is going.

3:29 Most people pay the $20 a month subscription to use Frontier models, but more importantly, the harness that comes with the subscription like Codeex and Cloud Code. And just like how switching

3:40 software is pain in the bottom region, ever since codeex and cloud code came up with more and more features directly in the harness, switching harness is also a big pain given that our conversation

3:51 histories, skills, workflow, connectors, projects, files, schedules are all in that platform meant to keep us tied into it. But the model layer that's grouped into the subscription is certainly being

4:02 challenged, which forces Frontier Labs to bring even stronger value proposition at the harness and the memory layer to keep users within their walled garden. Now, speaking of harness and working

4:12 with other people, it's important to find one that works for you. And that's why computer by DevRev can be a great option for team collaboration. Thanks DevRev for sponsoring this video. When

4:21 we think about AI agents, they're typically one-dimensional. Everything happens between you and your personal agent. But what happens when you want an agent that can work across an entire

4:30 team or an organization? That's the idea behind computer by DevRev. Computer is an AI teammate with shared context and memory, allowing your team to collaborate with it together in a group

4:40 chat. Instead of everyone using a separate agent, the entire team can onboard computer as if it was another teammate. Here you can see multiple people interacting with computer in the

4:49 same conversation. No matter who asks the question, everyone sees the same chat and computer can explain what it knows and where that information came from. Just like people learn through

4:58 experience and develop long-term memory, computer can retain useful context from its interactions with individuals and teams. Over time, it becomes more familiar with how your organization

5:08 works. For example, here one of my co-workers asks for lastminute pitch tomorrow and they can easily ask at computer to collaborate like this and computer will draw from past meeting

5:17 notes and customer profiles to generate this entire branded pitch and PowerPoint for our team to use. Pretty neat, right? You can also customize computer by connecting it to applications like

5:27 granola, Google drive, Slack, notion, and more. Computer is designed to change how we think about AI agents inside organization. It gives individuals the flexibility to personalize how they work

5:37 with an AI teammate while maintaining persistent context and memory access to broader team. Try computer today using the link in the description below. Looking at models like Quen 3.827B, 27B.

5:48 What I find really encouraging to see is the growing ecosystems in hugging phase that customize and help distribute the model in a way that can run on various hardware. Within just few days after the

5:59 release of Quen 3.827B, 827B. The entire community pulled together to release different quantizations of the model, already reaching well beyond 800 different variations of quantizations

6:11 from FB8, NVIP4, or even other variations like Obliterated or uncensored models to remove safeguards from the release model. So, not only are we moving at an incredible speed when it

6:22 comes to how fast the labs are releasing new models, the ecosystem that's under it is moving just as fast. Here at my own home, I can run Quinn 3.827B either on my dedicated GPU or on a DGX Spark.

6:35 And visually, the difference looks something like this. Using my dedicated GPU, I'm loading the quantized model directly onto my GPU's memory that gets streamed to my GPU registers at 736 GB

6:47 per second. And this setup gets me around 41 tokens per second running at home, as you can see. Conversely, running the same model on my DJX Spark, but at full precision, loaded on a

6:58 unified memory, and then having to stream the models weights to the GPU cores at a much lower 237 GB per second, gives me about four tokens per second, as you can see, largely from the

7:10 bandwidth difference between the two systems. One thing to keep in mind here though is that this really isn't an applesto apples comparison since I'm running a Q3 quantized version on my GPU

7:20 which is roughly four times smaller in file size than the full precision one that's running on my DJX Spark. The full precision is about 55 GB in my storage and the Q3 quantized version on my GPU

7:32 is at about 13 GB. So the 4.2 two times shrinking in size and placing them directly in the fabric of GPU helps me get 10 times the speed altogether in throughput which broadly the math makes

7:45 sense given bandwidth is about 3.1 times faster and the model is about 4.2 times smaller and collectively 13 times theoretical speed increase which is close to what we're actually seeing here

7:56 in practice. Now when it comes to different quantizations of coin 3.827b 827B model. Here's a quick chart from Atomic Chats where it shows you how different companies like Unsloth, LM

8:06 Studio Community, and GGML all contributed in taking the model size at a spectrum at 32 GB of file size and shrinking them all the way down to 8 GB along the way and the average KO

8:17 divergence in comparison to the full precision model. As you can see, the further down you go in quantization, the probability distribution tends to be further apart compared to the original

8:28 precision as the baseline, which in practice could look something like the model having different logics for each possible next token as they turn into different probabilities after softmax.

8:38 And in our case, the Q3 KM precision falls right at about 12.5 GB in file size, while the KO divergence is around 0.07, 07, which is the trade-off between these two setups. So, going back to the

8:51 narrative of cost, requesting intelligence from a data center versus running them at home. Since I'm generating at about 41 tokens per second, to generate 1 million tokens,

9:01 I'll need to be running this model for about 6.7 hours. My GPU pulls about 250 watts of power here in Michigan. My current electricity cost is around 21 cents per kilowatt hour for residential,

9:14 which is really high, I know. So that makes 1 million token output cost at about 37 cents per million output tokens in electricity cost. And in comparison to Core Reef that runs FP8 precision,

9:25 that gets about 17 tokens per second, which is slower at a much more expensive price at $3 per million output tokens. So quick math, I'm already saving almost eight times the cost and inference cost

9:37 running the model at home while having similar throughput on a consumer-grade graphics cards, which once again goes to show you how the economics for AI is changing with smaller models punching

9:47 way above its Guys,