Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Nexus is experimenting with AI agents as players in MMO RPGs to create dynamic, spontaneous experiences and solve the chicken-and-egg problem of populating new games, while also using agents for QA and game testing. The company has built its own internal development framework, Dainci, which uses multiple specialized agents and loop engineering to automate much of the development process, though they note that harnesses are becoming less necessary as models improve. They are also exploring self-hosting open-source models like Kimi to reduce token costs, and they see agent-optimized e-commerce as a near-term adoption area.
Key points
Nexus is researching AI agents as players in MMO RPGs to create dynamic, spontaneous experiences.
Agents could solve the chicken-and-egg problem of populating new games with players.
Nexus built an internal development framework called Dainci that uses multiple specialized agents.
Dainci uses loop engineering where agents report back and iterate until the task is complete.
Nexus is considering building an internal data center to run open-source models like Kimi for inference.
The company believes agent-optimized e-commerce will be an early adoption area for AI agents.
Tools mentioned
Techniques
- Loop engineering
- Agent-based development
- Model routing
- Self-hosting for inference
- Computer use for bypassing APIs
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
If you populate the the an MMO RPG with um agent players instead of NPCs, you're able to have a much more uh vast and dynamic experiences. The thing is, and again, this would be difficult, right?
They're relentless. So, a player has to sleep at some point. An agent doesn't. You can almost have undercover agents that are just testing the game for you to see and make sure that everything is
working. One of the earliest um adoption of Asgentic future I think would be shopping. So Asian has as we said Asian has a lot of context of myself. >> My agent usually just tells me to stop
smoking weed when [laughter] that happens. [music] All right, Isaac, you're working at Nexus. You all are doing a lot of stuff
with games. You particularly are working on the AI and blockchain field. I know you have a little bit of a lab going on there. You've got some cool skunk work things happening. Can you break down
what you're thinking about when it comes to AI in gaming world? I think there's been a lot of experiments across the globe from large studios to very small um indie developers using AI to develop
games. Um and one of the key things that I I was I learned at GDC this year as I was presenting was using AI as a companion within existing games to increase the retention of the users. So
really interesting thing that um it was a Chinese uh it was a I think it was a subsidiary of a Tencent. They presented a lot of metrics on how they enable um users to have a playmate within the game
that actually knows how you play. The AI agent has data access to to your gaming play and it's able to kind of help you become a friend in playing the game. So that really kind of struck me when And
that was like five months ago, >> about a half a year ago, right? And a lot of um interesting developments has happened since then. So for us at Nexus
um we're doing a lot of experiments from A to Z in we're investing into companies that are uh building AI platforms to create games uh from from just text to to actual serviceable games.
>> We're developing um simple tools to help users create games. For example, uh we're building we I think we recently built a tool called text to 3D. And the we give AI a preset of engines that it
could use in order to generate some sort of a physical world. And when you prompt the AI, you you tell them, hey, I want to I want to build a town. I want to build um some kind of map that's very
similar to some game in Roblox. and um AI is able to go find the right places where to integrate this engine, where to use this setup, and then it's able to create like a map that is that could
actually be serviced in in in robots. So, this was one of the experiments that we're doing. We're trying to push down on it to to to refine it and and I guess a lot of the people would agree that AI
helps you to bring the completeness of the result of of a product say by 80% 90% it makes you really it helps you a lot in develop in getting to that 90% but after after that it's a lot of final
work right so the you the users doesn't really care whether you use AI to devolve a uh right So um it's we're we're focusing on that remaining extra 10% so that the product that is built
through our tools and products are actually serviceable. >> Mhm. So it's not just that 80% done and then you have to work so hard to get that extra 20% that you give up and you
have a graveyard of 80% done things in your repertoire. I personally can relate with that one quite a bit because of how many things I've started and then that extra effort to finish it is just not
worth it at the end of the day or I don't have the time to hit it. I want to go back to this idea that you were talking about of using the using AI as a companion to increase the engagement.
And I presume the hard questions in that are making the AI at your level >> and it just doesn't go really really good and it it's not on like god mode and it's also not on really dumb. And so
you as a player, wherever you are, the the level it can kind of meet you at is where it should be, but then also you can interact with it as if you were interacting with somebody else that is
on the other side of that. And so you can tell it to go and do something or you can tell it that you're going to go and do something and that it should wait there and it will understand those
commands. >> Few few things I guess. Um, one of the research that's being done by Craftton, um, the operator of um, Battlegrounds, >> they're actually doing a lot of research
in this front. So, the the the the Playmate agent that that knows what you're doing in the game is able to kind of direct you to say, "Hey, you know what? You're you're kind of weak when
you're doing something. You know, you're you're kind of vulnerable. This is this is sort of, you know, based on your past play history, this is sort of where you kind of die out. So maybe
>> so it's not like another character. It's like a coach that's there that's kind of omnipresent. >> Yeah. Yeah. That that's one way to think about things. But if you think about how
AI how AI agent can be part of your um part of your play in the games, it really depends on what sort of game you're playing. for example, um if you're playing like, you know, um FPS
games, it could kind of help you, direct you, coach you. But if you're paying if you're playing something like um MMORPG, um one thing that we're actively researching is how to how to enable a
lot of uh play a lot of NPCs as sort of um agent players in the game. So um you know if we look at the MMO RPG landscape um if you if we look at successful MMORPGs obviously the game is
fun but when when we say the game is fun it retains a lot of uh users right it retain because it it retains a lot of users there's um competition within the users to spend in the game to get ahead
so it creates that dynamic um we're thinking that we might be able to bootstrap uh that specific phase with by deploying a lot of agents, Asian players in the game so that the human player can
come in and and feel the similar social dynamics they feel in in um natural human only games, so to speak. >> And why is that different? Forgive my ignorance, but
why is that different from just having traditional NPCs? Why is like infusing them with AI or having them be AI game players versus regular NPC
different? >> Regular NPCs everything's scripted, right? So there's no um spontaneity or you know if if you're so for example my interaction with you would be very
different than someone else's interaction with you. It's not going to be the same, right? Time and place is different. You know the thought that's going through your your head is going to
be different. >> Questions I ask you. Yeah, of course. um that's sort of the experience that we think could happen with um agent populated games. So um if we look at it,
if we look at the development of or history of of games, we have these studiogenerated content where whereas the studio the developers defined everything that you could experience in
a a certain in in a setting and that was it. Once you finished the contents that was it, right? And then the next evolution was user generated contents, right? You give the users an open world
and then they create this sort of you know additional maps, features um some sort of loops and users were able to generate the content right and I think the next um next phase of the
development is agent generated content within the game. So whereas if you populate the the an MMO RPG with um agent players instead of NPCs, you're able to have a much more uh vast and
dynamic experiences as you interact within that world. from day one for you too, as you were saying, you launch the game and it already has the tipping block or the tipping point of players
that you would need because a funny thing about this scenario, which is similar to a marketplace, is that you have the chicken and the egg problem. And so you're kind of passing over that
chicken and the egg problem. If you already have the game filled with agents, that can be spontaneous. M in those moments. >> Exactly. Exactly. That that that is what
we think will help the games to enhance their monetization and also to prolong their um um halflife so to call it. For every MMO RPG there's a certain set of um as the
game matures um the players are able to reach greater levels. A lot of inflation happens within the game. So there there's there's there's a lifetime value to that game. Um we're thinking that we
can increase that as well with use of um agents in the game. >> And would you give some agents the ability to have like some agents show up and they're rich and you can trade with
them or is it everybody all agents are starting from ground zero and they have to all work? I think I think the way we see it is that they should have the same restrictions of of of they should have
the same starting ground as the players. >> Oh, interesting. And they also have the same tests that they have to go through or side quests that they can go on and they can choose if they want to do that.
>> Right. Right. >> The thing is, and again, this would be difficult, right? They're relentless. So, >> a player has to sleep at some point. an
agent doesn't, >> right? But if you think about, well, if you look at the current landscape of MMO RPGs, uh most of them offer auto sort of um farming.
>> Uhhuh. >> So, they're able to go on um go kill monsters on on the auto mode. So, already already the players, you know, it's the the experience for players in
MMO RPG is that they could pick the segment that they want to enjoy. Um rest of rest of the time the player's character is on auto mode. So doing whatever quests, farms, can hunt
monsters, etc. It's already on auto mode. So that's why we thought that maybe introducing the agent concept wouldn't be as difficult as as as building something ground up because
>> from like you know I guess if you think about MMO RPG from like 10 years ago you have to spend a lot of time you have to spend a lot of time actually playing the game.
>> Yeah. >> Yeah. Doing all the hassle related stuff right but right now the experience already is at where um the player can just cherrypick the moments that they
want to play. M um and they're here to experience the the so to speak uh dopamine loop within the game. So they're they're they're here to pick those pick the best moments to kind of
play at. Um and I think with the addition of agents I think we can increase the length of such experience as to also um provide the variety of the experience.
>> So now that's just one subsector of how you could be using AI within the games. You did mention prompt to full game as one potential outcome,
>> but that 10% trying to make that 10% work is where you feel like you have to put in a lot of effort. >> How are you doing that now? Can you explain like how you're helping make
that 10% easier? Because I have the biggest qualms about the chat >> being that language is fuzzy. So, a lot of times when we want traditional guey options like a slider or we want to just
like turn a knob just a little bit or bring I think about histograms when you're correcting photo coloring or something. >> There's a lot more fine detail and
granularity to being able to adjust things. >> Yeah. >> In that level versus if you say, "Okay, make it pink." No, make it more pink,
right? Or make make it big bigger. It's not really the same kind of granularity that you can get big or grandiose. It's it's also very subjective in a way, >> right? So, I think um we're I think I
could give you two examples, right? Um one is um so Nexus recently acquired one store and we're expanding into the um application marketplace. >> Actually we we already took a hold of
application marketplace and one store has a lot of um different uh businesses and one of the businesses that it does is it it um distributes content. So, you're able to read uh comic books,
novels within this this app. And using this um existing IP, we thought to create um some sort of um short web dramas that users are users can um um watch using AI. And um the problem
becomes that you're able to you're we we we don't develop our own model so to speak but we use whatever model is out there um to the best of our ability and the at best you could generate probably
you know 20 to 30 seconds of of a video. Let's say that we want to build an we want to turn a novel into some sort of um video series, right? So, we need to we need to create maybe at least
hundreds if not thousands of these 30-se secondond clips and they have to it has to be very natural for for human eyes to to to see that the story is actually flowing through, right? Um, but the the
trouble is that these segments are tend to be different. Even though they're generated with similar prompts, it it it tends to be different. So, um, what we do is we give very specific instructions
on how to generate video at which point so I going back to your question, right? In order to refine the work, um, you have to give it very specific um, instructions. So as as you said make it
pink make it more pink you have to define how pink it is right. So um when we build any sort of um AI induced products we try to define as much as what AI can do so that it doesn't have
to assume much. We try to define say if we're building a video there's got to be some sort of human characters in it. we try to define we we we we give instructions so that the AI goes say
okay the first thing I'm going to do is that I'm going to create a video using this story um so first I'm going to build a scenario and within this scenario um after building the scenario
I'm going to I'm going to describe how the characters look and in this 30 secondond segment the characters are they got to be wearing AB and C this character when it speaks it has to be
looking at that way when it interacts it has to sound like this etc. Right? So um the way we we do this is to we try to generate as um we try to create a very high level framework of what the AI
should be doing and then um after that it's a lot of iterations and trying to fix that that trying to color in that last missing piece um so that when you use this um system to generate an output
it it it lives up to your expectations. This is kind of similar as to one one other tool that we built was to if you create um 2D pixel games. Um using AI to generate a pixel is it's not
that difficult. But when you play the pixel games, it's it's it it it's got a lot of motions, right? But when you try to make the lo motion look very natural to human eyes, it's a lot of effort.
It's a lot of human effort to make that happen, right? >> But using AI, um, when we first tried it, it was it was awful. But it was just it was like you you you built we built
the 2D character, it's it's moving through a lot of different map, a lot of different um settings, and it just doesn't look natural at all. So what we did was um we laid out what we want from
this character and what we hope to achieve and um we created um a loop where the agent goes around and round and tries to fill in that that that blank so to speak.
>> Yeah. So this again is u one other way that you can use AI and I love the fact that you're trying to incorporate it into many different areas within the game design. One area that we
haven't talked to that I feel is probably the most relevant right now is how the folks that are coding are actually using it. Like are you shipping with all of the agents? your coding
agents, you're creating those like walk me through what the engineers are doing. >> Yeah. So, um we built our own um internal development tool called um Dainci.
>> Nice. >> Yeah. And um it's got one of our AI engineers. Um he's been he's been doing a lot of AI work prior to chat GPT blowing out. Um and back then he was
focusing a lot on doing um AI uh to use to analyze and increase the marketing performances. Um but we've been doing a lot of research on on that front. Um so we use our own internally built
framework to ship product. Um and within this framework um let's say that I'm building um say a notetaker app, right? Um when I say that I'm building a note take so we we we go into this ID and we
say hey I want to build a notetaker app when we say that it it classifies that okay this is this guy is trying to build an app when what what do we know when we build an app so it it's got it's got
some sort of a blueprint on how to build an app um and when that happens it has multiple different agents that are we have backend agents frontend agents um the the agents that are defining what
sort of features that has to be in there. We have uh QA agents. So all these agents have different roles with different um they have different defined jobs, right? And then Dainci um we use
uh loop engineering within it. So there's going to be an agent that says, "Okay, this is the task that we're trying to do and this is what you guys are going to work on. When you're done,
report back to me." it it reviews it and then okay we still have some missing missing pieces and we're going to do it again reiterate reiterate so this is the sort of the process that we use to
develop um and a lot of the time is actually spent on designing um I'd say core architecture of the product that we're trying to build and and evaluating the output
>> so development process in itself has has been almost fully replaced by by AI. Um and you know like half a year ago deployment was deploying
any sort of product service was kind of it there there used to be a lot of human effort that needed to kind of that was needed to push the product um out but even but now it's it's just fully
automated. So, one thing that I have my agents do when I'm building is when before I push something, I will ask it to go through and it's almost like a QA agent and I ask it to write a report
with screenshots >> and click every button, click every link inside of whatever I'm building to make sure that everything's working properly. And so, it writes a report. It has
screenshots. that says yes, this is the correct page. This is working properly. And then it'll click a button and it will say okay, this is actually going to the right place. And if it if there is a
broken button or there is something that's not rendering properly, usually in the screenshot, it will catch that and then it can go and it can kick off another sub aent to try and fix it or it
will fix it itself and then u push that through. What came to my mind as you were talking about this bringing that original idea of having agents playing inside the games, you can almost have
undercover agents that are just testing the game for you to see and make sure that everything is working because you said that a lot of your time now goes into the evaluating. Yeah.
>> I imagine evaluating is playing the game, >> right? Right. >> And so how does the evaluating happen? And is it just you have uh you have game
players there, you sit down on the weekend after you push something and you play it and or is there like more of a mapped out way? >> Interesting. Um we haven't we haven't
actually applied this this QA concept to to games yet. Um not yet. Not yet. But I think I think that's definitely an interesting method for us to try out. Right now we do human QAs. Um obviously
a lot of a lot of I guess basic stuff has been abstracted by AI but still um the thing that we're optimizing for is the user experience. So um I think it it it still requires human human touch to
say that you know this is good this is good to go out. Yeah, there is a little bit of magic there that is that it's beyond that 10% that we were talking about earlier
>> when the game is prompt into existence. You still need some something that will hook the person and that you can't really get even if every pixel is in the right place and everything moves the way
it should be moving since it's a game you need that game play. >> Yeah. You need that human experience. Games are games are for humans I would say for now.
>> Okay. So you could try maybe it would work on certain levels. >> Yeah. I think I think um I think doing so would help us to maybe uh find minimal bugs a lot faster
>> before it's reported by players. So definitely I think there's an upside to it. >> Yeah. >> Yeah. We're going to try that out.
>> Yeah. Yeah. Thanks for the >> You're going to have to tell me how it goes. I do love the idea of like some of these you fill the game with agents like you were saying and some of them are
just undercover QA agents and they're reporting back like hey fix this and it kicks off another agent loop and it gets pushed and so you have that that type of thing. It
>> would probably be a much bigger report than my agents make with my little dinky apps. you know, if you have a whole world that these QA agents are reporting back on and taking screenshots of or
making sure it works. >> But I I do find that fascinating. So, you've set up your own development environment. You chose not to use some of the ones that were already out there.
Was that just because you started so early? I think it's partly that we we also research a lot of existing open uh frameworks and then we want to kind of handpick the features that we want in
development. Some of the things aren't really necessary to us and and I guess a lot of a lot of um effort in in designing this sort of development process is so that we can continue to
refine it in a way that is um excuse me um that is that is right for us. what's right for us might not be right for other sort of um development companies say you know if you're developing a
financial app what they are going to be doing their development process is going to be a lot different than ours right so um I think that's the kind of the core reason why we decided to develop our
develop and maintain our internal framework another thing is um security right so if you use external frameworks Um it's it's always yeah the security team always wants wants some sort of
closed systems >> and am I right in saying like this framework in a way it's like you've built a harness for agents >> right
>> and so this is fascinating to me because harnesses are were very popular to be building feels like a a few iterations ago. >> Mhm.
>> Because of all these reasons that you're talking about, you want to have that controllability. You want to be able to use it and and make sure that it understands your use case. Have you
noticed as time goes on that it becomes less and less relevant? Because I think now it feels like the industry is moving in the other direction where they're saying, you know, what's the best
harness? no harness or like the minimal harness that you get with pie and that's that's the good enough one, >> right? So I think I think harness is is it could be a a it could be a very
little thing as like you know um whenever you're prompted A, you should go look at skill B and then you should try to um do things in C order. That's a very that's a harness, right? And it
could be as complicated as as as anyone wants to be. But before before harness was kind of a thing. We we we had our agents we we we used these sort of methods to define to tell AI that you
know your job is AB and C. You should do it in this order etc. This was like the initial sort of harness that we used. And back then we had like um 200 different um sort of agents role
defined. >> Um but frankly as the models got better and better this is it becomes a lot it becomes not necessary. So, um, harness it. It could it could be very
useful when you know exactly what you want to do. Um, and you want agents to um kind of fall into this repetitive system. Whereas doing so would get would would um help you guarantee some sort of
quality of of the work because if you do it in a certain way, the the results are going to be the outputs are going to are going to be similar. So um from from that point of view harness is useful in
making developments um in in developing a product. Um but as you as as as as you said um I think as the model gets smarter um I think harness wouldn't would not be necessary.
>> Yeah. >> The importance of harness would decrease over time as models get smarter. >> Exactly. the harnesses become thinner and thinner and they're just there to
really >> help you figure out. I I just bring it up because I remember what was it like a day or two ago I saw or I was listening to a podcast about how exactly what
you're saying we used to have to define every agent. We used to have to create every tool >> in order for it to use. But now it's like, oh yeah, just give it to
Claude or Chatbt or Codeex and it will use Bash. >> Yeah. >> If it needs to or like the It's It's wild now how much we've moved away from
that paradigm and needing to really like say here's the tools that you have. Here's the and if you give it GitHub, it's got infinite tools type thing, right?
>> Or you create your own tools. So, and you are a QA agent. You have these five tools, but now we don't necessarily need to do it that prescriptive and we don't need that many different types of
agents. >> Right. Right. I think codeex and and Claude, they're they're a harness, right? So, they're they're a very very good form of harness. It's I think if
you try to like deviate away from that, it's because you have a specific need that that these >> say I'd say general harness is not kind of it's not it's not doing something for
you, right? It's not that these are bad. It's that these are very general, but say that you know we want the agent that's developing the game to do a certain sort of um um work that that's
sort of where we would use harness. >> Yeah. Are you guys using a lot of open- source models as well as the closed source labs or do you >> Yeah. Well, primarily we think we are
currently developing with um Frontier Labs models. Um I think I think they're definitely um a bit better. They used to be much much better than open source, but I think they're still a bit better
than open source models. But >> um since last month, um I think what did we see? The Kimmy >> Kimmy models. Yeah, they blew it up. But how are you doing? I always like asking
people like how are you accessing the Kimmy models because if you want to >> run it on your own hardware, you need a few GPUs lying around. >> So, we're actually building that.
>> Really? >> Yeah. After after we saw the Kimmy news um last month, we you know, we we as a development company spent a lot of tokens every month.
>> You're not the only one. >> Yeah. Yeah. Um and you know when we look at news like that and see you know if we could reach 80 90% of the work that we're doing at what onetenth of the
cost. Yeah we got to look into it. So uh we've started to look into um look into building a small internal data center so that we would set up GPUs and then run it on our own.
>> Have your own rigs. Yeah. >> That's very cool. that is uh where in theory it's 10% of the token cost but then you always have to remember like how expensive is the engineer that knows
how to work those GPUs and the people that know how to like hook up the GPUs all of that stuff is is part of it which I think at the end of the day if you amortize it over a long or depreciate it
over a long horizon then you're going to come out on top especially especially because you can continuously swap out the models as they can get better and better and so you're not stuck there.
And these like these GPUs you're seeing that even A100's which are I don't know how many generations ago. It's so it feels like they're ages ago the A100s came out and it was like wow that was
amazing. >> Yeah. >> You could still use those. Those are still very good machines, right? like you could put a lot of model onto those
machines, especially if you get a few of them together. So, if you're to buy a few very beefy machines today, they're going to last you a while. >> Yeah, this is something that we've just
started to look into. But I think from from what we've gathered so far is that >> we're not looking to train anything on these GPUs. So, we're trying to inference. And for inference, I think
we're looking at at least five five years to use GPUs. >> So, um, if we spend X amount on ex tokens that are currently going to the frontier labs, it I think it would make
economic sense for us to set up and then just self-s serve our needs. I guess one um one concern that I have is that is um open source labs, they could always choose not to open source their weights.
Yeah. >> At some point in the future, right? >> Meta kind of did that, >> right? >> They said, "Oh, yeah, we're llama fully
open." And then they went and they closed the doors, but it seems like they're going back out to the open again. >> Mhm.
>> So, it is a it's definitely a valid concern. >> Yeah. Yeah. But even even then, I think it might be worth I think it's worth for us to give it a try.
>> Yeah. >> Yeah. >> Yeah. See what happens. And I I get the feeling that you'll be able to use the machines well longer than 5 years
because as time goes on and as you can start, we're already seeing that the router and the the gateway type of pattern >> for model routing is a very common
pattern. And so as the machines get older, you can just put smaller models on them >> or what today is quote unquote small maybe.
>> And then right >> as [clears throat] you get then you get more machines and you can so you can continue after 5 years with those machines that you have today. They're
just might not be running the best and newest and greatest models on them, >> but it doesn't matter because they are of a certain quality, >> right? So the model can or the router
can still route to them when they're needed, >> right? >> I find that stuff fascinating. It is a huge investment. I know some friends in
Switzerland are doing it right now >> and the biggest headache that they've had is just like getting and setting up the GPUs. GPUs are very finicky. Mhm. >> They're very They don't play nicely all
the time and you really have to get down to a low level to do that. So >> interesting. Um what what are they trying to build it for their own internal needs?
>> Yeah. >> Okay. >> Yeah, exactly. And it's they they have it downstairs in their basement and heating their whole building basically.
And uh it's it's one of those things where they have some incredible engineers on it, but Nvidia doesn't necessarily they're not so open with how GPUs work that well. Like you you
program in CUDA. >> Uhhuh. >> But if you really want to know what's behind that, like what's lower than CUDA type thing, that's where I think it
starts to get a little bit obscured. >> Okay. Okay. So from I guess running that software on GPU um it might require more knowledge than programming on CUDA and that's not really not it's not really uh
common knowledge. So >> yeah, especially if you want to I it's not common knowledge but also if you really want to optimize like the the GPU for what you're trying to do. I
would say though, there's this great book. I actually interviewed this guy, Chris. Uh, I'll have to find his last name and put it in the show notes because
>> he wrote an amazing book on like optimizing inference for for this specifically. And he went through like, hey, I he reverse engineered it because his friend he had
a lot of friends at Nvidia, but they wouldn't tell him. they they can't tell him. I guess they've signed pretty hardcore NDAs, but he's able to poke around enough
>> to reverse engineer it. And then he he wrote it in this book and he kind of gave his >> all of his secrets. He shared all of them at least as of
>> when he wrote it, you know, a few months ago or >> I I'd love to get the name of the book. >> Yeah, I I'll share it with you for sure. Thank you. It's uh it's a great one. The
other piece on inference >> specifically is you've got VLM which is a killer open source product and that helps a ton for running >> these models and on the GPUs
>> and I know there's different companies that have worked really hard at making that even faster. >> Oh wow. And so you can hopefully find stuff out there that will um that will
get you like super fast inference >> on top of this, you know, lower cost. So there's all the GPU providers now, right? Like there's the uh
Neoclouds is what they're called. Yeah. And some of them have different value props like I look at have you heard of this come like modal or base 10 these neo clouds
[clears throat] their value props like modal's value prop is you can fire up a GPU really fast so you can go to serverless and then you can go out of it and wake up the GPU in like seconds as
opposed to minutes. >> Okay. And so it's just really fast and they've done a lot of work, really, really hard work on the kernel to be able to boot it up that quick and load
everything in and get your model and your data, everything that you need. They're ready for you. And then you've got base 10. And from what I've understood from them is they did a lot
of hard work on the inference side. And so they said, "VLM is great. We're going to optimize it even more." And so they've written like this Frankenstein version that is very fast uh of that.
>> That's awesome. >> Yeah, there's there's all kinds of ways that folks are trying to optimize all of this. And when you start rolling your own, you get you go down these rabbit
holes. You know, there's a really good um community. I think it's called like GPU mode and they basically just talk about GPUs all day and this stuff. >> Definitely something that I need to look
into. >> Yeah. If you're going to if you're going to >> you know pay the >> price of a GPU
>> Uhhuh. >> or a few of them. >> Uhhuh. >> Then you should find the the communities in the literature where people are, you
know, talking about this stuff. >> We'll do. We'll do. >> Well, I know you've been working on a ton of different experiments in the AI lab.
>> Can you walk me through a few of these? We already talked about Da Vinci, which was one, right? >> What's >> What are other ones? Right. So, one
other one is called uh text through 3D. I think I talked about this briefly. Um where you prompt a text and then it's able to build you some sort of a world and within that world it's able to build
you structures and then in this structure um you're able to kind of go into say you know I want to play an FPS in this map. then it's going to give you the abilities to go into the um first
person kind of settings and then you you'll be able to shoot around. >> So it integrates multiple different uh physical engines and AI is able to help you build this sort of uh environment.
So that's that's one thing that we did. >> You've got like something that is a an engine where physics works. >> Yeah. It's not that we develop our own
engines. I think we integrate existing engines to call upon it. So for example like >> let's say one of the one of one of the reasons I guess what the background on
how we started this was we were thinking of um creating uh sort of like a map editor for Roblox. A lot of people build on Roblox and if you try to actually build on it it's it's a lot of hassle.
You have to learn how to use it. Mhm. >> Um, and the existing AI enhancers are not that useful, right? So, we we try to we try to use the physics engine that is provided by Roblox and then, you know,
>> help creators build games um for Roblox. >> Um, so that's how we initially started. uh we used some multiple different engines to kind of um do a PC um and as I said it it it's it's you you integrate
let's say that you have to first define what you're going to enable creators to do what tools you're going to be able to help them with what well sorry what tools are available and then
>> and tools you mean like which engines not like which tool calls the model can make >> right Right. >> Well, I guess in a way it is kind of a
tool call if you you have to tell the model to use you know when you create a world think about this specific construct or this specific um construct say that you know
when you when you're building a city you have to have a b and c um and whenever you're and then that would be the sort of um baseline of where we start and when a character interacts within that
um environment, that world, um it should have these sort of um capabilities. So, it it's very similar to a tool call. Um it's just that you have to you have to lay out what you what the model has to
do >> in order to achieve the desired outcome. That that's one thing. Um another thing that we built was a short form generator. So, as a game developer and
publisher, we have to market our product, right? So um one thing we built was that you're able to so the the model is able to watch the video and then create reverse engineer the prompt for
it. You're able to create input um any sort of image along with um prompts to generate a video. So it's it's it's even it I guess it it helps you to create pretty okay videos with say like two
sentences of prompt and a few snapshots of the game >> and videos like gameplay video >> so that you can showcase what is possible.
>> Right. Right. >> Wow. So that that's one thing that we built um that we're using it we're we this is a product that we're using in production to market our our current our
last game our current game that we published like a month ago. >> Wow. Um so that's one thing and then um we have we built our own internal meeting node uh meeting note taker we
call it Bob >> and one of the things that it does is that you're able to turn it on and you you talk it it runs on local model >> it runs on on the local local GPU of the
computer and then um it it it it's able to kind of whenever let's say that we're discussing about um some theory some idea and like GPUs right we're saying that you know a00 H100 etc what model is
at least how much does it cost you know if we run infants on it >> what's that going to come out to Bob is Bob takes notes on these and then it's able on on a side chat it's able to hey
can you verify that it's going to it's going to go on and verify that it's going to make another call to a model to verify any sort of important things that we're discussing real time
>> so um we think that could be very handy. It's not it's not fully um we don't fully use it. It's still under development. We don't fully use it within the company yet, but I think in
in a few weeks I think it's going to go out. >> That is cool. >> One other I think extra thing that um we're doing is uh it's a product called
>> Sorry, just one other thing that came up when I think about Bob. Yeah. >> Is a lot of times I talk with the same people, >> especially at work.
>> I would love if Bob knew who I was, you know, like so that it doesn't have to so it has context around me as a person. What's my job? What do I do? What are my tendencies? I like my notes like this.
>> Bob, when you send me notes, send them to me in my style because this is related to my job. And then maybe a sales guy who's in a call with you guys, >> he likes his notes in a different style.
So send him those notes and make sure that like each individual person or each individual team, right, >> has their own style that they get from Bob,
>> right? So you're asking for personalization. >> Yeah. A little bit more personalization. Interesting. >> Yeah. And especially around because I I
got this actually. This is quite nice if you have a new notetaker and you really want to make sure that it understands everything and it has the full context. >> My friend Fausto used to before we would
actually start the meeting, start the meetings, he would make us introduce ourselves >> because then the notetaker would have that context. And so if you don't have
to do that or so you don't have to do that every time every time you have a meeting, you can have the notetaker just remember that like >> yeah I think that's that's yeah
definitely something that we're going to we're going to try out. Thanks for Yeah. So I think we could build um say a profile of everyone that's in the meeting based on the discussions and
then we could also you know tell any sort of profile to Bob. >> Exactly. bop would build um outputs based on the profile. >> Yeah. And you could potentially have
this just with some I mean if you wanted to get wild, you could create it. You could embed different chunks of what folks are saying and then have the um the profiles and have it in a vector
store and all that fun stuff. And at the end of the day, you almost are like building a recommendation system on top of Bob, which would be very cool, but you don't probably don't even need to do
that. You just create some simple file system >> that it's like, hey, this is Demetrius's file, Bob, whenever you write like invoke the Demetrius skill that explains
how Demetrius likes to get his stuff back. And here's a few like short sentences on Demetrius. >> Yeah. Yeah. And so you can take it to like two extremes. I don't know.
>> But I think I think that's kind of the future that we're going to live in pretty soon. Whether whether or not we build it, I think we're going to see these sort of products with AI. You
know, I'm sure I'm sure you'd agree is one of the things that's very very critical for AI is is giving it context, right? So if we if if someone finds a very um
a very good way of delivering context of the things that we're trying to get it done as a company. Um I think that would be very interesting for us. >> Yeah.
>> Not not just for us but for a lot of the a lot of the a lot of people out there. Yeah. >> 100%. And the thing that I think about is like how much context is too much
context >> because on my system at home I am it's fully untethered and I'm like whatever you need you can have it. >> I the more data the better. So
>> like I notice you're wearing an Apple Watch. I normally wear um one of those health bands. >> Mh. >> And if I can, I want to give all of that
data via the API to my system also so it can just like go and make a daily report. Oh, check this out. I see that when you were in this meeting with this co-worker, your heart rate spiked. Were
you angry? Were you happy? What was it? You know, like so you can reflect back on that type of stuff. >> That's interesting. But at work, you may not want that. Like, you don't want to
give all of that health data to the job so that they can see like, dude, you get stressed every time you're in our standups. What's going on? you know, um I saw I saw an exp post some time ago
that was there was a woman that recorded all of her um all of her dates >> and then she uploaded that to Claude and analyzed it >> and that was it was it was kind of
getting a lot of heat from from the fact that she was recording it etc. But like she was able to analyze her her dating partners based on that interaction. But you know I think
>> it does get weird. That's why I'm saying how much context is too much context cuz you can take it a little too far in these Well, who am I to say that it's too far, right? Like
>> we're exploring right now. What is too far? Is that weird? Is it not? I don't know. >> Yeah, we don't know. We don't know. Well, that that actually might be the
sort of sort of the um future that we might have to get accustomed to. Whereas it's not just human that's making any sort of decisions, but >> there's always some sort of um
>> some sort of watcher or AI observer that kind of that kind of studies the interaction and tries to tell the human to iterate based on that. Um not sure if that's bad.
>> I don't know either. I I know that it gives fear to a lot of people. Mhm. >> I remember when I was first getting into college, university, >> photos, like taking photos of people
>> was not as common place as it is now cuz it's on our phones, right? >> So, you had to have a camera and you had to >> actually whip it out and then take a
photo, right? >> And it was >> more of a thing >> to take a photo. And a lot of times you would get caught in the back of a photo
and there was like then photo bombing became a thing. You would jump into other people's photos. Now if I'm caught in the back of somebody's photo, whatever. Like I that probably happens
to me a hundred times a day. Back in those days it was like, "Oh no, I'm those people are taking a photo. Let me get out of their photo range." You know, if I did that today, I'd never get
anything done. There's so many photos being taken all around. And so it it kind of feels like that like we're going to get more accustomed to >> having less sort of privacy, I guess.
>> Yeah. Or having more AI in our life, more context that we're given the AI. Who knows? Maybe that's a dystopian future. Maybe not. I like how you're saying it could actually be useful for
us. >> Yeah. >> Depends [clears throat] on how you use it, right? >> Yeah. Is there another experiment that
you had in there? >> Um, we did a marketing. We built the marketing agent. So um it helps you to um analyze the presence of your um product of your say
game a product a service whatever whatever whatever you're trying to push um it it it it it's connected to the SNS mainly as X and then it cause all the discussions on how your product is being
discussed and how people are talking about it what sort sort of how you should position your product. It gives you an analysis. It helps you to push it. So it also runs on a loop. So it
helps you. So it's got a few different agents. An analyst agent that analyzes your product says, you know, maybe the things that you should position for are A, B, and C. And then it's got agents
that collect the data and say, you know, you're only mentioned X times uh per day. You should try to get it up by doing A, B, and C. >> Yeah. It's giving you advice.
>> Yeah. Yeah. Where my mind goes on that is not necessarily on this product, but I think about how I used to pay for API access to some of these or for example like LinkedIn, you can't even get API
access if you want it. >> And so it's it was notoriously hard to get that kind of social listening for a platform like LinkedIn. But now I just use computer use. And [laughter]
>> so it's like you bypass all APIs, which to me is like, oh man, they're going to figure out like that I'm doing this and there's danger there, but >> until they do, this is quite useful.
>> You can you can just bypass the API. And granted, it's much slower. >> It is a lot slower. and you so if you're doing it on a scale that you guys are probably doing it on, it's a much
different thing. >> Mhm. >> But it is a a way to try and bypass like the LinkedIn >> policies.
>> But I think I think going forth like not providing an API would probably mean that you're not It comes with it's it's a double-edged sword, so to speak. Um you're going to get hit by a lot of the
agents. >> Yeah. But if you're not being hit by the agents, you might not get traffic at all cuz like you know if you like even for little things, right? I think I think
one of the earliest um adoption of Asgentic future I think would be shopping. >> Yeah. >> Yeah. So Asian has as we said Asian has
a lot of context of of myself what I eat you know my health data what I do and then you know let's say that if I was searching for you know my eyes are a little dry I want some you know eye
drops what should I buy and agent is going to it's going to call via API and then figure out what the best thing is for me and then say you know >> my agent usually just tells me to stop
smoking weed when that happens. [laughter] But I guess that's the personalization we were talking about before, right? [laughter]
>> But no, I I was thinking about this a lot because Perplexity got sued by Amazon for their shopping agent and it was very much doing just using computer use and then getting everything on
Amazon, >> right? >> And Amazon said, "Hey, you can't do that." And so they sued. But recently, I think I saw that the court ruled that it
is okay for or for Perplexity to have a shopping agent for you >> uh on Amazon. And so Amazon can't they like have no grounds to sue at least at this point in time,
>> right? >> And that's a huge loss for Amazon >> because now they are not getting that first party data from their customer. Well, they are kind of getting it, but
it's through a agent. >> Yeah. >> And then second of all, everything on Amazon or any e-commerce website for that example there, it's been hyper
optimized for humans. So, you have the popups. Hey, save 20% if you give us your email. That's a human thing. Like we do that because hopefully it gets you to buy or they upsell you with, oh well,
if you get three more dollars worth of products, we'll give you free shipping. >> And those things work on human psychology. >> But for an agent, it's like, "No, I just
want pants. I want pants. I want pants. I don't care about no email. I don't care about the save $3. I just was ordered to get pants." And so we have to think about optimizing now for agents.
And what does that even look like? What does that even mean? Because an agent is tasked with doing something. It goes right directly and it does it >> or it gives you 10 different options.
>> And one of those options is Amazon, another one is just the random e-commerce store. >> Yeah. Yeah. So I think I think going forth like as you said optimize should
we optimize for for human users or should we optimize for agents and and optimizing for agents >> if I had to guess is I guess providing as straightforward info as possible
>> um or if you're able to kind of retrace the origin of the agent right now um I guess the most of the well all of the API doesn't kind of authenticate on who the calling party is. You have no
information on who the caller is, right? So perhaps I think from a product or service perspective that might be an interesting way to gather information about okay so you want access to some
sort of functions or calls. Who are you? >> Yeah. tell tell me about you so that we can give you a set of >> the right things that you're looking for >> or we can show you some sponsored images
also >> because that's another piece that like where's that going to go all that sponsor revenue or the the ads revenue inside of a e-commerce store like
Amazon. >> Mhm. >> This has been great, dude. I appreciate you doing it.