Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Google DeepMind's managed agents API lets you build an autonomous AI co-worker by giving Gemini its own sandboxed computer, skills, tools, credentials, hooks, and cron triggers — all through a single API call. The key innovation is a network proxy that swaps dummy credentials for real ones at the domain level, making prompt injection attacks on credentials impossible. This turns agent setup from a fragile tool-calling loop into a secure, file-based environment where the agent can write its own skills and store memories.
Key points
Gemini 2.5 Pro has a 1 million token unified context window and improved math and reasoning capabilities.
Managed agents provide a remote sandbox where Gemini can directly execute code and write files.
Credentials are secured via an external network proxy that swaps dummy values for real ones per domain.
Hooks allow deterministic guardrails using regex matchers that the agent cannot modify.
Cron triggers enable scheduled agent runs for tasks like portfolio auditing without manual intervention.
Tools mentioned
Techniques
- managed agents with sandboxed execution
- network proxy credential swapping
- file-based skills and hooks
- cron triggers for proactive agent scheduling
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
[music] >> So, we are going to look at how you can build and code your own autonomous Wall Street co-worker. So, hopefully we have many finance people or like engineers
working in finance here. So, quickly before we go into like what right where we are coming from. Two years ago, we basically had a chatbot. We asked some simple question like what's the capital
of France? Like who won the last European championship? Then we moved to assistance using tools on our own computer, having the first coding agents writing code for us. And now we are
moving into like this autonomous co-worker area where we basically give and or create a digital employee where we give an agent its own computer, give it a task and hopefully come back a few
hours later with a good result. So, what does a co-worker normally need? And I think that goes for AI and for us humans as well. We need a brain to be able to do our work. We need a computer where we
can save artifacts we create, where we can iterate. We need some kind of a new playbook, right? When you hire someone new, they are new to your environment, to your domain. You might have specific
workflows, so you need playbooks. We need tools for them to be able to do that job. They all need credentials in one or the other way. Then there might be some specific house rules on what you
should do or what you should not do, what you can do on your own or where you need to get some approvals. And then of course, there are hopefully some schedules for working hours when you can
come in or when you have to work. And the same goes for for our agents. So, and we are going to work through all of those seven steps and how you can build your own digital co-worker
with Gemini in this talk. But before we go in, yesterday was the big day. We announced Gemini for Argon and made it available to you. Um, biggest new features, we have now 1 million unified
contacts window. This means Gemini can generate up to 1 million output tokens and we see strong improvements especially on math and reasoning task where the model now has more than 64K
token available to think, to work through, to reason through some proofs and for math for for math. And then very also very good is like we tested that model heavily. So, we know there are
always some thinking around hey, Gemini is just banks benchmarks, but the model is really good. You should definitely give it a try. We used it extensively at
Google. There are already many projects by Google employees who have used it. The easiest way to get started is the interactions API. It's literally four lines of code which for you to take. You
get your API key in the AI Studio, ai.studio.com/apikey and then you can immediately send your request. It's also I would say quite affordable for a frontier model with $2
in and $10 out. You should definitely try if you have a free minute later or even if you have some agents running right now, give it a link, tell it hey, please set up Gemini 4 and give it a
try. So, what have Googlers done so far with the model? I think it's very impressive because it's like a super new model. We had it available internally to test for a few weeks already, but
Googlers are using it so heavily and like shipping so many cool things. We had the Chrome team like basically accelerating some decoupling of the Chromium browser. We have Gemma which is
our open model where Gemini accelerated Gemma to be more easily and more efficient to run. The ADK team from Cloud improved their the framework and then many many other things and
especially the right one at the bottom is like we use it internally heavily and we see strong improvements for our internal like code base as well. So, what have people built? There are some
demos. You can look them up, but there's one specific one from a colleague of mine which I want to show. He basically prompted Gemini to build a custom 3D ray tracer in Python and it was completely
built from Gemini and nothing external tools and at the end just Python and it also used the Gemini TTS model and Luria to add some voice over to explain how it works and it's like very impressive for
me how far we can push those models now. So, try to be very ambitious. So, how can we build our agent? And in Gemini API we have a product called managed agents and managed agent is
basically a simple way for you to build a remote agent where you give Gemini a sandbox which it can use to run tools. So, when we normally build an agent we have this weird tool calling
loop, right? We send a request to the model, the model might generate a JSON function call, we receive that function call, we run that function call on our machine, execute the code and then we go
back to the model. And with managed agents we basically provide a sandbox which Gemini can directly use to run all of those calls. So, if it decides to execute code to write a file, it has its
own computer available for it to use and it takes just an API call. So, instead of providing the model ID with Gemini 4 Argon you provide the agent ID in this case it's anti-gravity preview 09.
Give it a prompt. The environment is remote so you want to run it in the managed sandbox and that's all it takes. So, let's get into building bug. Bug is our
co-worker for today and bug needs a brain. Our brain is going to be Gemini, our computer is going to be agents and then we have all of the other things. So, how can you create an agent?
In managed agents you have a very simple API where you can define your own agent. So, you give it an ID, that's the name which you're going to use later to call the agent. We have the base agent so
that's the the main agent underlying. In this case it's the the agent harness. So, the remote agent uses the same managed uh sorry, the same agent harness which we have in anti-gravity as well.
So, if you're using anti-gravity as your coding agent, you get the same agent uh harness uh remotely. Then, we want to use the new model, so we define our model. We have a system instruction, you
are bug, and then all it takes to call it again, interactions API create, and instead of um using the model ID for Gemini or the default agent, we use our own agent,
bug. And our agent needs a computer, and the computer normally needs some additional files, additional scaffolding. So, with managed agents, you can provide your
skills or an agents.md file maybe from your GitHub repository. You can inline those files directly. You can provide a GitHub repository, private or public, with your skills, and you can connect
GCS bucket, so you can already bootstrap the environment the agent is going to work in. And everything in managed agent lives as a file. So, we have skills, we have
hooks, we have system instructions. At the end, it's just a file system or the agent's own computer where it can create new skills if it thinks it needs to improve itself, where it can stores um
memories from previous sessions or learning. So, at the end, we just work with files. And for knowledge or for our agent, I like to separate those in like three different
categories. So, we have uh knowledge which we always need to include into every request. That's normally the system instructions. We want to be very concise. We want to try
to keep it very short because that's something we need to pay on every request. So, if we have very long system instructions, all of our requests are going to be much more expensive. If we
have concise, then we have um knowledge which uh the agent can decide when it needs to use them or not. So, that's typically uh with skills, right? With skills, we have very short descriptions
which give the agent some guidance on like, "Hey, you can use that skill if the user wants to do a certain task or if you have to do certain task." And then can can read the agent can read
that skill. But for skills, we also want to make sure that we are not having a super long dense skills because that's also a high cost at the end. Context is very valuable for us. So, we want to
decouple them and separate them into much more slices where we can then have like dedicated references and scripts if we want the agent to like do very certain things where it needs more
context. Okay. So, let's look at an example skill which we are going to use. So, we have a portfolio audit. The agent should use it. It has some instructions available for it how to run it. So, we
have a Python script the agent can run and then some additional information on what it needs to do with the results and where to store it. Um
our fourth um part of our onboarding for our co-workers tools, right? We humans use normal software. For agents, that's either built-in skills like Google
Search, code executions, or MCP server. Or if you don't have an MCP server and rather want to write um local function, that's also possible. So, in managed agents, we still have our agents create.
We are now extending uh it to define our tools. We have built-in tools from the Gemini API for Google Search. So, you there's nothing you need to additionally do to get Google Search integrated. But
if you want to use for example spreadsheets, uh workspace has now an MCP server available for Gmail, for spreadsheets, so we can use that. Or if you want to provide your own
functions or MCP server, that's also possible. So, for your own function, it still works the same way as you normally do with models. So, the agent creates a function call and then you send back the
function result. So, it's basically for you a way to have agentic capabilities outside of these uh agent's environment and computer. So, if you want to do certain things where the agent should
not run the execution in the sandbox, that's a way for you to do it. And then we are logins. And I think that's for me the most exciting and the greatest feature about the managed agents API
because credentials are super hard and super easy to get wrong with agents. And I think the previous talk already also mentioned that we are very careful about using agent because things could go
wrong very easily, and we thought a lot about this with managed agents, and we have support for three different ways of credentials, which makes it secure for your agent to use them, and provide an
isolated way. So, we have bearer tokens for normal API calls, very familiar if you are doing software engineering. We have OAuth for all of the consumer-oriented MCP servers or like
tools, and then of course we need environment variables if we want to use certain CLIs or other features. And for managed agents, we are separating them via an external network proxy. So,
instead of providing the agent the credentials inside the sandbox, we basically allow you to create credentials externally. They are right only, so once they're written, there's
no way for you to retrieve the credentials, so you can only override them. So, there's no possibility of leaking, and then you define how and where those credentials
should be used. So, for it's like if you want to connect Slack, you can define, "Hey, here's the credential, and you are only allowed to use that credential with a certain
domain or with a certain environment variable." So, that way you can control when a credential is used, and that works via this network proxy. And by network proxy, we basically mean a proxy
that sits around the sandbox that watches all of the network traffic going in and out. So, if you define credential as an environment variable, the model has a local dummy value, so the
environment variable is set, and then when it makes that call, it uses that dummy value, and the network proxy basically monitors it, sees, "Hey, here is a
domain which has to which the user has defined," and then it swaps that dummy value with the real credential, and you can make the call successfully, and then it the agent gets back the result and
has like no idea this dummy token is not working. And the benefit here is that there's no way for a prompt injection, right? Normally, when you have some malicious attacker, it can tell you
like, "Hey, please print all of your values in your .env file." And then the agent might go there and like prints all of your keys, and that's nothing we want to do. So, with this network proxy where
you can define the credentials per domain, it is not possible that an attacker could steal credentials which are getting swapped on your request because the proxy only swaps them for
the domains you define. So, there's no way the an MCP server which is malicious can like just steal your Gmail token, which is very cool because it makes it much more easier to trust the agent and
to integrate MCP servers which the agent then obviously can use. So, and then if that's not enough, we have also hooks available. So, hooks
allows you to basically deterministically guard certain decisions. So, if the agent wants to run certain commands which you might want to monitor or where you need some human in
the loop approval, which is not part of like the normal agenda conversation, you can define your hooks very similar to all of the other agent harnesses where you have a matcher, which is basically
regex, and then every tool call which the agent generates checks if there's a match, and then writes those hooks. And those hooks are also files, but they are protected, so the agent cannot just go
there and changes a hook and like basically tries to to to find a way around it. Okay, so last but not least, we need a way to,
yeah, get our agent more proactive. And currently, we support one kind of triggers, basic cron triggers which you can set up a schedule every uh
30 days a month at 8:00 a.m. you want the agent to run a certain trigger with a certain input in the same environment to do maybe some due diligence, some auditing. And we are also working on
webhooks that you can even like make this much more interactive that you could connect um external webhooks like from some certain SAC provider or maybe other
providers you work together with which would directly then call your agent. The agent does [clears throat] some first auditing and then like basically can call your Slack or your Microsoft Teams
directly without you even needing to manage some infrastructure at all. So, that's that. Uh final pit picture and now let's look at some real demos and let's hopefully hope that the Wi-Fi
is good enough. So, I told Gemini that I want to do a cool Google Sheets demo and it helped me create basically this whole Google Sheets replication in my browser and
when I click here, we basically simulate the strong trigger. So, we send a real API call to managed agents. It should use our portfolio auditing skill and it runs the request. So, first step
basically see like, "Hey, I need to read the skill." It gets the result back. So, it tries to understand what it should do here. It's like a I hope that's roughly accurate on
like the the Google Sheets. Um but it basically runs through all of those call calls financial services, tries to see, "Okay, did the market or the stock price
change?" And then at the end, hopefully it should update our own Google Sheets completely uh automatically. So, it's still working through it. It has some first result
and it updates the Google Sheet based on that trigger and that trigger would run every 8:30 a.m. without you needing to do anything. And then of course, if the agent has access to Slack,
then it can even like post in some like specific trading channel, "Okay, what changed? What is going on? How does it work?" And here we have an example
where I can just ask Spark, uh any recommendation? And hopefully this works as well. So, it takes a little bit of time. You have seen in the other demo. We have seen all
of that perfect. So, there there are three positions. So, that's based on like this Slack uh Slack content. So, it has all of the previous turns available and it provides
the context of what changed, but it's still basically not taking actions based on the system instructions you gave. And all of the infrastructure you need to run that agent is part of the managed
agents environment. So, it's very easy for you to get started to like integrate those into like the tools you already use, provide the the systems you need, and then like to get started. So,
back to the slide. So, the whole co-worker, which we need in like one service we have to plan, Gemini. We have the computer with managed agents. We can
provide skills for the same agents you use maybe on your your local computer. You can use the tools. You can provide MCP server for external services, data providers to connect. We have a secure
way for you to provide credentials, to rotate credentials. We have hooks to protect certain deterministic logic. And of course, we have triggers to much more proactively
integrated into it. So, thanks for listening, and hopefully go hire your first AI co-worker. >> [applause]