Think You Can Build a Game with AI? Think Again! - Danielle An & David Hoe, Meta

[music] >> Hello, thank you for joining us before lunch especially. So, I appreciate it. Uh, I'm David from our AI for Meta. I'm Danielle. I'm a principal engineer at Meta. And we have a little set of slides here. We thought we'd make a make a little different experience today. So, instead of non-interactive slides, you get some interactive slides. So, if anyone's interested, you can just scan this QR code. I'll try and make it bigger for you. And uh if you get fidgety, have a go and see what this does along the way. This is linked up to by the way. So, instead of slides, each NPC will reveal the slide content. So, depending on how chaotic you want things to be, you have control. Everyone got that? I'm going to close that down. And over to Danielle. Yeah, so while you guys are getting the hang of this, it seems to be working. Well, multiple people are controlling it. There's a voting system behind the scenes that will determine who wins. But if you try to yes. It's like a little quest-giving game, but before we go any further, the reason we're doing this is that the two of us are leading some of the AI-driven gaming creation work at Meta. We've been doing that for over a year now. And a lot of what we do on a daily basis is using AI to help generate games for the casual creator, pro creator, and everybody alike. So, given that that's our full-time job, we figured that we will take you on a little bit of a journey on the games that David made in just the last 12 hours. So, now we're going to get started. So, because there's lunch and the previous talks are a little bit delayed, so we appreciate you being here like David was saying. But we should also be honest about why you're here and we want to tell you what kind of value we might bring or not bring. So, we're imagining if you come to this talk, you're people that are very interested in games, maybe you play a lot, or at least you're somewhat interested in how games are made. And even better, if you wanted to make them in the future or if you made them in the past and you you're wondering how AI is going to change that. So, this is the kind of talk for you. But if none of that is interesting to you, at least you can just have fun play the game before lunch and work up your appetite. But um now we're going to get going on to the real part. In 2026, a lot of you probably have either built something like a little game, like a platformer game, or a Tetris or something on Gemini. If you have not, maybe your kids, your moms, your friends have built them. Very chances are it's a very likely. And >> [snorts] >> initially, it's very impressive cuz within a couple prompt, you're like, "Oh, here is a game. I can I can recognize this." And if you're lucky, it works out of the box, but not guaranteed. But the problem is that as time goes on, you would all see that the novelty kind of wears off and a lot of what you prompt is the same. If everybody says, "I want a little platformer game that looks like Mario." Overall, they all kind of look similar if they work at all. So, after a while, it's like, "What's next? This is cool." So, that's um Yeah, we're all moving a lot here. So, people want to see what the text actually is. >> [laughter] >> It's good to just kind of not move for a while, but I don't know how many people are here. Actually, I'm not sure how many people are here, but a lot of people are controlling, so it's hard to stabilize, which is a part of the game. But the point here is that everybody can build a game over the weekend, but what is the next thing? And why What does this mean really to the industry? And like in our opinion, what this means is just a lot of people previously who wanted to build games, but were gated by skill sets. For example, I'm a coder, but I'm not an artist. I cannot do 3D modeling. I cannot draw. I cannot get art. But now you can just use Meta Banana or Mesh you or whatever else to get yourself unblocked. Um same thing with artists who previously really wanted to build games, but maybe they don't have the coding skills and so they have to wait for engineers or they just feel like that's not possible. For a set of people that really want to utilize this moment to build more games, if the barriers are removed, but then that doesn't mean everybody can make a good game. What will make What it will separate a good game versus a bad game is still a lot of the typical stuff. And here we're going to share with you some of the tips that we've learned in the last year working on AI game building, which is how do you stand out? Um I hope so far you're not too bored. How do you stand out? Um here's a few things. Like one is usually aesthetics, right? Like after you prompt a few games, you just you're not satisfied with like a little box that's purple next to a box that's green that shows like a platform. There's a lot of art to that. There's also a lot of cohesion to the game you make between the UI and the stories and then art. Is Does it feel like one entity? Does it feel like it's one lives in one universe? And then there's also lots of like play tests and like the real people with real feedback who tell you what is good. But um just going to directly go to the video that maybe David can comment on what what's the point of this video. Yeah, so this is the a video from one of our art directors that we work [music] with, Dale. And he wanted to demonstrate here example workflow that you can try out, which is really about using key art [music] as a an anchor for the fellow models. So, just like game development, you might have a stage where you're iterating and you come across like a a concept that you really like. So, same thing with the models, you can [music] use just a single key art image for game for inspiration. In this case, you can see the key art was a lovely bear example. [music] And the difference here is that you go from being able to, you know, um It >> [music] >> basically anchors you on um having a an image that you can use for the art style that you can then filter down [music] to the assets that you can see here before, but also it can also help anchor on, you know, what the [music] gameplay could be. So, it actually holds a lot of information and it's a very simple way to get started and >> [music] >> allow for the LLM to have some cohesion throughout the sessions as well. So, yeah, that's just demonstrating that. So, this hopefully shows you that what the difference between something you can prompt or your kids can prompt versus a professionally made game. And the distance is shortened by a lot because what David is saying, all the AI tools that can help you with that. Um but another once you get to the basic level of like, "Okay, this looks like a decent game." One of the most important things is what will make your game stand out if every game looks polished? What makes your game the best, not just better? And what makes it the best is the same as everybody know in the AI age would say is the taste. What it really means to the games is that it's not that you delivered a game, but it's that you have the feel that this is a game humans would like. And which subset of the humans would like and why they would have fun with that. Like what are Um And then once you reach that level, you're like, "Ah, great. Now I can make a game." Now, the problem is that not quite because now there's introduction of new technology, runtime LLMs. Um what that means is that while you are playing the game, there is a living entity, the runtime LLM, that is modifying, changing the game, directing the game in some way. Here is another example of that. Would you like to comment on that? Our team built this again just over a couple days and this is a multiplayer game where each of the NPCs here is entirely driven by LLM. So, this is the part where the humans are setting up a competitive game. It's a four four players playing against each other. >> [snorts] >> And by specifying what the NPC should do, giving it a personality, maybe it's a thief, maybe it's very honorable, maybe it's very fast. Whatever personality it has, now the game is about to begin. And you will see these runtime LLMs independently making decisions, achieving the goal of this game, which is to get as many cubes as possible. But as you watch, you will see some of the NPCs will make decisions to steal other NPCs' cubes or block other NPCs, kick other NPCs. These are entirely not scripted. These are runtime LLM-driven decision-making. So, it just adds a lot of spice, a lot of dynamic um ness to the games. Every game is unique. Every game is not repeatable. And so, that is a new form before the introduction of AI in the last let's say 18 months which is now possible. So now we're sort of experimenting a lot with this genre of games where you can see these NPCs starting to fight with their personalities. You can obviously design the game to be more spicy um or more friendly, co-op or competitive. And again, the point of these games is that they were previously not in the industry made it was just not possible to make them, but now the inference time is fast enough the models are cheap enough for us to actually build these kind of games. Again, build over a couple days. So next um we're going to share some learnings from us driving these teams in the last 12 years uh 12 months. Feels like 12 years now. Um directing teams um across disciplines and what have we learned using AI tools and how does it change the game building experience? Um briefly the first thing it changes is in the past building games are very expensive because it's a linear kind of process. From one department to another, first you design and then you go to like art, then you go through modeling, potentially you go through animations, you go through coding, but all of this is kind of like a linear waterfall model. And what that means is that once you commit to making a game, it's very hard to revisit the decisions upstream. That's very very costly. But nowadays because of AI we have achieved teams that are working in parallel and updates and iterations on the games are achieved in hours, days as opposed to months. What this achieves is just that games are more fun because you have more play test. You have more um time to iterate on the game idea as opposed to just linearly executing and this is a game changer. And the second thing that we've learned uh on on top of the taste it's just with LLMs, with runtime decision making, it the there's a game master that can help you make the game more personal, more interesting for you. For example, for myself, I'm not very um I'm coordination challenged. So when I play games with my friends, I'm always if it's co-op, I'm always the one that is like tanking the team. And in the past, it just sometimes stops me because I feel like I'm just dragging the team behind. But if LLMs can adjust that for me, allow me to still play with my my team and have fun, that is big win that that previously was not possible. Another learning is as we previously said, runtime LLM is going to be a game changer. But just like any piece of technology the technology itself does not make the product better. We are just like a lot of the teams still at early phases of exploring how do you use runtime LLM? We imagine a lot of these kind of things like Star Trek, right? You say, "Imagine in front of me there's a forest and now I'm hunting bears." And then it just suddenly happens. We do see that eventually becoming reality, but it's not currently the reality today. So today how do you use runtime LLMs to drive your game to be differentiating against other games is still something we're exploring and a lot of people are very interested in this area. But in general, there's lots of challenges. I think everybody, whether you're in the game industry or other industry, what do you do feel is that every day AI is changing what you're doing from the tools from to your code base to what good looks like and just keeping up with that has been very hard, very challenging um but also very interesting um if that's the kind of thing that you like, that's going to take you a really long way. Um It's refusing to talk to the NPC now. Um And the last bit I'll talk about this kind of technology driven is now you imagine we work at Meta like Meta cares about being a platform of millions and millions of piece of gaming content that will come online especially now that everybody can create. One of the main challenges is just that through all of a agentic systems from the front end where the user is prompting potentially to runtime the LLM is making decision like the game we showed we showed before to the platform serving content, ranking content delivering content. Through the whole platform there's a an agents and agentic systems at play. So that just means and determinism throughout the whole stack which is something that as engineers, especially at scale, we are struggling a lot with in general, right? Because we're used to thinking that you write code, you write test, stability, debugging against a set known a known set of code base is what we call stability and scalability. But now in these days, if your model is changing, model upgrades, if your prompts is changing, a lot of things can in in entirely throw off your system. So how do you even debug that? How do you um engage with systems like that is still challenge. But since we have only 2 minutes left and it's about lunch time let's say something like that is that inspires David and I on a daily basis is that this just means that for the whole gaming as an industry is being transformed is being transformed by AI and we're day zero. So for those of you who have read or seen Ready Player One, this is the moment where we think we're at the beginning where that Oasis can actually be a reality. So if you want to be part of that you're going to I do really hope you guys navigate to her cuz there's something very cool here if you navigate to her. Hello, please. >> [laughter] >> Uh Come on, please. Can we go to her? Ah, yes. So I personally love Lububu. Um so this is the one of those personalization moments where with AI you can steer any game towards anything that you want. And for me building this moment just means that you can create a universe of Lububus and you can be any Lububu you want, you can collect any Lububu you want. And last but not least, a little bit takeaway before lunch is wherever it is you are, maybe you've never built a game. So then the the task here for the takeaway for you is the weekend is coming. You can just use any of your favorite models, maybe Gemini Manas, whatever it is, just try to prompt a very basic game like Tetris or um a infinite runner, something that you play and just see what what it does. It's going to fail, it's going to bring you delight when he surprises you and it works really well. Or maybe you are the people that already went through that phase and that's why you're in this talk where you're like, "What is the next phase?" The next thing for for you is probably something we mentioned prior is to upgrade your game with art, with UI, with surprises so so it doesn't feel like it's something that is prompted out of the gate so you can upload there. But if you're beyond that as well, then if you're engineer that cares about scalability, then think about tokens. How do you make sure creators, players, and platform with token economy are going to be profitable um and content safety is one of the bigger challenges is how do you make sure with runtime LLM, maybe potentially generating images and content, you know that the content is actually safe for your audience. There's a lot of open challenges there. Whoever it is you are and I mean negative time. So thank you very much for being here from me and David. Thank you for the participating and working with the NPCs. Hopefully you had a little fun and go to lunch. Thanks. >> [applause] [music]