Meta's Open Weight - Muse Glimmer 30B

Okay, so in what is shaping up to be a huge week for open models, kicking it off is one that many people didn't expect at all. And this is Meta returning back to releasing open models. And this particular open model is called Muse Glimmer. So I covered the original Muse model back at the start of April. It was a very respectable model. A lot of people kind of bagged on it, but I kind of felt that this was the first model from a team that was put together late last year. And the interesting thing here is that wasn't even sort of a full team. So since they finished training this model with the pre-training, they were actually able to convince other researchers from other labs to join them. One of the big coups that they pulled off and didn't get reported that much is they actually brought over the head of reasoning from the Gemini team. So this is actually a few months back before all the huge exits from Google had even started. And sure enough that team has delivered with newer versions of Muse Spark, not to mention their own image models and video models. And that brings us to today where they're launching Muse Glimmer. So Mark Zuckerberg himself announced Muse Glimmer along with the same time announcing that they're actually going to release the weights for the bigger Muse Spark going forward as well. So while we're not seeing the return of Llama, it is awesome to have Meta back in the game contributing open weights. And in the case of Muse Glimmer, those open weights are actually coming with an Apache 2 license here. And you can see it's even kind of cool to see people like Yann LeCun who left Meta in response to this super intelligence lab that was created by Alexander Wang, even he's congratulating them here. All right, so what is Muse Glimmer? So this is basically a 30B model. It's a dense model, not a mixture of experts model, so it's not going to be super quick. And really, if I'm being succinct about this, this is their answer to the Quen 3.6 27B. And you see, sure enough, if we come and look at the benchmarks, they're benchmarking really against the Quen 3.6 27B. So, they're beating the Gemma 4 model pretty easily in most of these, but it does seem that the idea here is that this is basically a model that can replace Quen 3.6 27B. Now, don't forget we're just hours or days away from Quen 3.8 27B coming out. So, it could be that Meta was just trying to get this thing out before they having to benchmark against the new 3.8 model. That said, the benchmarks are pretty impressive in here. We can see that they're not beating Quen 3.6 on everything, but they are on a lot of these benchmarks. The other thing here, too, though, is that this is the first open model like this that they've released. So, you've got to imagine that they're still working on the best ways to do post-training, they're still working on the best ways to do distillation, etc. Now, in their blog post and their report, they've got a bunch of interesting details about how they actually trained this. So, it's really nice to see them coming back and at least giving us some of the information in here. So, it looks like this has basically been a combination of distillation and training on the larger Muse Sparks output for pre-training, which is kind of interesting if this hasn't actually been trained on sort of, you know, raw internet data or cleaned internet data like most of the models are using now. And we can see when we look at the post-training here that this is a very familiar thing that we're seeing, a combination of on-policy distillation and reinforcement learning to make this thing better. And that's one of the things that we've seen recently with the Thinking Machines Inklings small model. And I've sort of done a video about that which I'll try and finish up and put out at some point. The big difference here though is this is with a much smaller model and a model that people are going to be able to run locally. It's also interesting but not surprising to see that Meta is developing this model for agents, right? They say this straight out in their blog post and we see that in a lot of the benchmarks as well that this is basically something that is going to have multi step reasoning, tool use, the ability for doing long trajectories, the ability to work with different harnesses, whether they're sort of coding harnesses or things like Open Claw, Hermes Agent, that kind of thing. The other thing that has really changed since Meta was in the Llama business is that now out of the box people expect you to release a quantized version so that they can run this locally. And sure enough Meta is not disappointing here. They themselves have made a quantized 4-bit version here and they've made it specifically to be able to fit on 24 gig or 32 gig cards, right? So that's very clearly aiming at the sort of 3090, 4090, 5090 kind of cards or things like the AMD 9700. And you can see that they've sized this model so not only does the model actually fit but they're leaving quite a bit of head room for you to be able to have a decent size KB cache going in here. Another thing as well is that they've clearly got the memo about speculative decoding here. You can see sure enough they've got a D-Flash speculative decoding enabled in this model and you can see that the demo that they're actually showing here is running on a MacBook Pro with 64 gig of memory. That just wouldn't have been the case when we were talking about the Llama models in the past. So you can see here the kind of token speeds that you're going to get if you're basically using it on a Mac versus if you're using it on something like a 5090. And my guess is that over the next couple weeks we'll see many different quants of this. So, if you want to get started on this, the model weights are actually out on Hugging Face already. You can be one of the first people to try it out and install it. And while many people are going to think that the big question here is whether this is still going to be relevant on Friday after we've seen the Qwen 3.8 27B, the thing I would say is most important here is that this just shows Meta is getting back into the open weights game. The fact that Zuckerberg's already said that they're going to release weights for Muse Spark 1.2, which is going to be really interesting to see what are the quantized sizes for that model and how is that actually going to perform. And don't forget this is coming in hot after them releasing Muse Code, which is their coding version of the Muse Spark model. Now, we don't know if that's going to be open or not. Hopefully, eventually it will be, but all of these releases are really signaling that Meta is definitely back producing top-tier level models. And once the Muse Spark 1.2 weights are out, I'll definitely make a video about that and have a play with that and show what that can actually do. Already, Artificial Analysis Intelligence Index is actually showing that model to be on par with Claude Opus 4.8. And being able to run that locally is going to be a huge win for a lot of people. So, anyway, stay tuned. There's a lot of releases coming out this week. We've got things from Qwen, we've got things from Deep Seek, and we've got some other models coming out from big companies as well. So, let me know in the comments how you feel about Meta being back in the open weights game, what you think of Glimmer 30B, and as always, if you like the video, please click like and subscribe, and I will talk to you in the next video. Bye for now.