Build Your Own Team of Managed Agents - Jack Wotherspoon, Google DeepMind

0:01 [music] >> Hi everybody. Thank you so much for coming. My name is Jack Witherspoon and I am a DevEx engineer at Google DeepMind and today we're going to talk about

0:20 managed or cloud agents. So raise your hands here, who all has heard of either cloud or managed agents or has used them? Okay. I'm starting off on the right

0:32 foot. So you might have heard of them if you for those who haven't, cloud or managed agents are essentially a way of instead of running a harness and running coding

0:43 agent on your own machine, you can actually just run them in the cloud on a sandbox. So let's first start by why the heck would you even want to do that? Well, sandboxes are really powerful for

0:56 a number of reasons. Number one, they're not running on your own machine. So you can, you know, spawn a cloud agent. It can go and write code. It can reason. It can do research,

1:07 search the web. It can go ahead and write tests, run tests, verify and actually come back with verified kind of answers. And that's a really strong thing. If

1:18 you've ever like tried to run verification either on your own laptop, you know, sometimes it steals the browser. It starts to run things for you. You start getting pop-ups and

1:26 things like that. So running this all in the cloud takes away that pain. And the other reason is that your laptop doesn't scale.

1:36 You might start to have specific setups like agent skills or you maybe have customized your personal coding agent on your machine. But then how do you go and share that with a colleague or how do

1:48 you integrate that into your app, right? You don't want your app that you're deploying to customers to be running with your own kind of subscription or things like that, right? So, we want to

1:58 be able to scale this up and doing so. And managed agents or cloud agents are a great great way to do that. So close, the sign almost stayed up. Don't worry about it.

2:09 >> [laughter] >> So, when it comes to kind of managed or cloud agents, you know, to do this yourself is quite a tedious process. You know, you have to actually spin up a

2:21 container or a VM. You have to load things onto that, manage it. And then you actually have to do harness engineering, you know, figure out kind of build your own harness within the

2:31 sandbox. And then let's not even start to talk about networking or setting up firewall rules or all the security stuff. So, that's where kind of managed agents

2:41 come in and whether it's Gemini managed agents, cloud managed agents, we'll be talking about Gemini managed agents today, which is our offering in the Gemini API at DeepMind.

2:51 And really now it boils down to just being a single API call. So, that is really the beauty of managed agents. With a single API call, you're getting all the things we talked about.

3:04 So, you're getting the sandbox, you're getting the anti-gravity harness built out of the box that is in the sandbox, and it can go ahead and do a lot of powerful things.

3:13 You don't have to worry about managing any of the infrastructure or the scale as you need in your application more agents, you can just do more API calls.

3:26 Actually, we'll go back for a sec. So, you can see here, this is our interactions API. It's our kind of new agent first version of our SDKs and client libraries. So,

3:37 previously we had kind of models, you would, you know, do an input, get an output, but that didn't really scale to agents in the new world we live in. So, we launched a new kind of SDK uh,

3:50 covering interactions. So, interactions are much more kind of mirroring what agents are doing. So, it can be multiple steps, you know, when a when you do an agent call, it can go ahead and write

4:00 files, call those files, call different MCP tools. Um, so really this new interactions API that we launched, uh, and went GA earlier this year, is now our kind of home for managed agents.

4:13 So, you just call the agent instead of having a model, you have an agent. So, we have our base anti-gravity agent. And then you set environment equals remote. So, that's that key thing that's now

4:23 telling it, "Okay, we want a managed agent that runs in the cloud." And then you give it kind of regular input as you're used to. So, here's kind of how the

4:33 infrastructure actually looks under the hood. So, when you call a managed agent, you kind of have that input that we saw. And then it calls the Gemini API, which has a bunch of, you know, built-in

4:43 server-side tools like, um, searching the web through Google Search or file system tools to read and write files. Um, and you can also add your own custom functions or MCP tools as well.

4:56 And then we have the sandbox. So, that's actually running on kind of Google Cloud infrastructure. Um, so when you call the Gemini API to get a managed agent, it goes ahead and actually sets up and

5:06 spins up that sandbox for you. And you can go ahead and actually upload files to the sandbox. So, if you have agent skills or agents.md, or you want to load a GitHub repository on startup, uh, we

5:18 allow you to do all of that cool stuff. So, here are kind of the building blocks that make up a managed agent. These are kind of the different core pillars. Um, so you've got the harness itself, and

5:31 that's the anti-gravity harness. It's the exact same harness that's running, uh, in our anti-gravity suite of products over at Google. So, that's what's kind of figuring out

5:40 the tool calls. It has the built-in tools. Um, and then we have the environment. We'll talk a bit more about how the environment is kind of its own entity,

5:49 um, which allows you to do really cool kind of multi-agent workflows by having different agents pointed at the same sandbox so they can write files. A different agent can pick that same

5:59 environment up later and actually build upon those files. And then we've got skills we talked about so you can upload your skills to the sandbox just as you would have potentially like playbooks or

6:09 different ways of doing things at your company or on your personal agent, you can upload those to the managed agent as well. And then we have credentials.

6:17 Credentials is a new piece of managed agent which essentially lets us inject either kind of tokens or your MCP OAuth into the managed agent sandbox so you can access your different services.

6:29 And then we also have triggers which allow you to actually schedule a managed agent to run. So let's say at 8:30 8:30 a.m. every morning before the market opens you want to go and you know

6:40 do some kind of financial analysis before the market opens up. You could set a trigger. The managed agent would essentially wake up, run its task or run its input, output a

6:50 result. You can like do webhooks so that you could send a Slack message. Maybe it queries your team's Jira board every morning and sends you know here's what everyone's working on

6:59 and things like that. So we talked a little bit about the harness. It's the anti-gravity harness that runs under the hood and you can change the

7:08 different models that it runs with. Yesterday we just launched Gemini 4 Argon which is our new frontier model. So I definitely recommend testing it out.

7:17 It's a great model. But we'll show it later on in this in demo. And you can set the different model in the config for the managed agent in the

7:26 agent config as you see there. We have that kind of one-line parameter to set the model. So that's how you can kind of tune the managed agent based on the workflow. If you're wanting to do

7:35 something that needs to be fast and efficient, you can use Gemini Flash. If you want to do do that's deep reasoning or potentially doing a lot of analysis, then you can use a different model like

7:45 Gemini 4 Argon. Like we said, Gemini 4 Argon was released yesterday. Um it's a really great model. It's especially good at financial tasks, which is great for

7:56 hopefully for the audience here. Um and really good also at doing data analysis across a wide range of documents. Um and as always, the Gemini models are also super super good at multimodal.

8:07 Another really cool, interesting um feature that we have with the Gemini 4 model is long decode. So, normally models have an output token max of around 64K tokens. Um but with Gemini 4,

8:21 we actually have up this so that now the 1 million kind of context window or token limit is now shared between input, thinking, and output. So, output, as you need to kind of scale up if you need to

8:33 do extreme kind of math reasoning or financial modeling where the output might be pages and pages and pages long, the model can actually go and do that. And this is showing to actually advance

8:43 some of the more mathematical and financial benchmarks out there. So, we talked a bit about the interactions API and environments, and this is kind of one of the key pillars.

8:55 These are kind of two core primitives in managed agents that allow you to get a lot of flexibility and be able to customize kind of the agent configurations that you do.

9:05 So, these are kind of two different um stateful things on our back end in our server side. So, interactions is what manages your conversation. So, when you put an input and get an output and kind

9:18 of a different turn with the model, um those are saved as kind of what we call interactions. So, you can take one interaction, pass that ID to the next interaction, and that kind of, you know,

9:27 it's the second turn. Um and then we also have environments. So, environments are what's actually kind of in the file system on the sandbox.

9:37 So, when the agent goes and writes files or creates images or does different file tasks on the actual sandbox in the cloud, that's kind of representative of the environment. And

9:50 you can go ahead and use our APIs to download files from that environment or upload them. But having these two things kind of split out allows you to do some really

10:00 cool things where you can actually see here that we are creating one agent interaction with our first code block. And it's going to go ahead and do some work and save a file. And then we can

10:11 pass that environment ID to our next agent. So, it actually picks up. It has a very fresh kind of context window and conversation, but it's reusing the environment. So, it can go and, you

10:22 know, say save the Q3 filings in our workspace. Maybe we're going to go, you know, fetch the fetch the annual reports, the filings. And then we're going to go and do some analysis, save

10:32 that as a file. And then we can have a new agent come in, reuse that same exact environment, but this time have a fresh context window. Kind of a multi-agent workflow, which is

10:44 really cool. So, there are a bunch of built-in tools already in the sandbox, like code execution, reading and writing files, as well as Google Search. But we also

10:54 support, you know, remote SMTP servers, so you can bring your own servers. You can hook up to your favorite services. And really that takes the agent a step further and be able to actually do kind

11:03 of daily tasks or things that you can build into your apps. And we talked a little bit about it, but you can also kind of mount files to the file system. So, you can upload your

11:15 different agents.md files, your skills, and kind of run through the playbooks that you have your agents doing on your personal machine, but doing it in an environment where you can kind of

11:24 reproduce and have that work done for you in the cloud. So, you don't have to worry about, you know, overriding any files or messing up things on your machine. Everything runs in the cloud,

11:35 um and can just run as a process and get you back that output. So, here's a Here's a code sample where we're actually going to upload an agents.md example into our um sandbox,

11:48 and then we're also connecting a GitHub repository. So, now you can actually have the agent look at your code, check for bugs and things like that. And then you can also save the agents.

11:58 We also have an agents API, so you can do agents.create. So, once you've kind of refined your workflow and found an agent that works for you, you can save it so you can easily call it, you know,

12:08 in multiple different times. You can iterate on it and then save it for later. And then you can just easily invoke your personal agent by calling it with its uh

12:18 agent ID. So, another big part of the security aspect of managed agents is that secrets never actually enter the sandbox. So, our managed agents have a proxy that

12:30 sits uh it's a kind of a layer between the sandbox and our API. And it actually what it does is it puts a placeholder kind of dummy token into the sandbox so that the agent can never kind of go

12:43 rogue and say, "Oh, I see that I have this GitHub token laying around. Let me just see what else is on there." Um so, it can actually never do that because it never actually has your GitHub token.

12:53 So, what it does is when it goes and calls the GitHub API, our proxy will then inject that token. It will match it to say, you know, it's API uh GitHub API.

13:02 Um we're going to go ahead and inject this secret. Um so, that way the agent can never do anything that it shouldn't be able to.

13:12 We talked quickly about triggers, but triggers are also really powerful because you can have them set to kind of a daily briefing every morning like we said, scan your emails, maybe suggest

13:22 ones to reply to, or if you want to check on the stock market before it opens, um you can do that. And this is a way to run one of your agents on a set schedule.

13:37 And we talked a little bit about multi-agent workflows, but managed agents can actually work really well um in multi-agent workflows. So, you can actually set it up so that one managed

13:47 agent knows how to, because it's just an API call, actually call other managed agents um and do work and fan out work. And then you can have very kind of niche

13:58 and specific targeted agents that have one core purpose. They do that purpose really well. Um you know, maybe it's drafting notes or doing research. Um

14:08 writes it to a file, gets that information saved, and the next agent can then take over from that different input. And this kind of breaks it up so that you can have things really

14:16 controllable and really make sure that you're doing specific tasks well instead of just giving a super super broad, you know, task to an agent and having it have no way of kind of measuring what

14:28 it's actually doing. So, now we'll just quickly put this to the test. Um so, we can actually run the agents here and see what this actually looks

14:39 like when you run one. So, we have an SEC kind of an SEC filing researcher that just researches companies. We have it set to Nvidia here. And we can go ahead and make call the

14:48 API. So, like again, this is just an API call. It's really easy to build into your own either internal workflows or to actually publish in your services or apps to external folks.

15:00 So, we can go see here. There are step traces. So, each thing the agent does um is saved as a step trace. So, you can see we're having function result for the

15:11 right to file, code execution, and you can kind of trace the what the agent is doing and optimize as you go to kind of make sure that the workflow is straightforward. And then were we using

15:22 the same environment ID that we talked about? So, because we're using the same environment ID in this scenario, we're actually having our agents pick up. So, we can have We had this SEC filing

15:30 researcher that went and did research on Nvidia, gave us a bunch of metrics as the output. And then, we can now kick off this kind of quant agent, which will go take a look at the metrics because

15:42 it's in the file system. It's reusing the same environment, the same sandbox. And now, it can pick up its task and its job from that um metrics.json, take it, run a bunch of Python code files to see

15:55 and actually look at it make evaluation for Nvidia. And if we go here and look at the MD file, we can see the managed agent went and actually did research based on the

16:05 metrics it got, and it's now kind of doing a forecast for the share price in the stock. And then, we can kick off a third agent just to kind of take that research and

16:17 knowledge dump and turn it into a report. And if we had connected different MCP servers, like we could connect Google Docs, have it actually output and write to a Google Doc, or

16:25 send a message in Slack so that you can really kind of create these customized workflows from managed agents. And then, you can also build managed agents super easily with the SDK into

16:38 your own application. So, this is just a demo here where we have a managed agent that is actually um connected to our service. Um it's connected to our Google Drive where we dumped in a bunch of

16:48 receipts of our annual kind of um company data, and it's going to go ahead and actually create this visualization um by from from the the data. And the managed agent is the one that's actually

17:00 doing all the research. It has, you know, kind of a skill that will go out and tells it kind of how to do this interactive report. Um and then, it's just kind of reasoning through it and

17:10 building this out for us. So, it's really customizable. You can really kind of anything you can do on your own personal coding agent, you can kind of turn it easily into a managed

17:24 agent that you can either run as a company daily or set up in your own apps so that we have other ones on our on our Gemini API on AI Studio where you can kind of create games on the fly.

17:35 You can research documents and do all kinds of things. So, the easiest way to get started today is to go over to AI Studio and get started for free over there. And we also

17:51 have our documentation for managed agents, but I'll quickly just do another demo here since we have a few minutes. This is one of our kind of managed agents examples that is on AI Studio in

18:02 our gallery. And this one's kind of just trying to showcase how you can actually build these managed agents into your own applications.

18:10 So, this is you know, create a nice front end and put in a stock. Oh, not slash. So, we can go ahead and do VOO. And this ticker agent is a bunch of our different

18:22 managed agents that will go out and do specific tasks and look for different analysis on a stock. And again, this is kind of more of an illustration illustration that's like as

18:40 you're building managed agents into your own apps, you might not want to have this kind of stream but just to kind of visualize what the actual agent is doing. It's reasoning

18:49 through all the different steps in the interactions API. And then this managed agent is told to kind of generate a report based on all of the findings and reasoning it did in

18:58 its sandbox. So, now if we go ahead and click on the report, we can see the managed agent outputted this kind of PDF file for us where we can see all of the research it did, see its analysis and

19:11 key takeaways, as well as doing these charts so we can visualize. And it's also going to go and link out to kind of all of the resources

19:20 um that it went ends actually using for this. So it actually went to the Vanguard Vanguard site, found the different documents, read them in, and

19:29 then did analysis on it. And that is it on managed agents. Thank you very much.