Managed Agents - Don't Get Locked In

0:00 Okay, have you noticed that over the past five months or so, almost every major AI company has shipped a product called manage agents? Alongside people shipping new models, alongside people

0:12 shipping new frameworks, we've got this whole new area of managed agents. So, Enthropic was the first one to kick it off. They started it in April or even perhaps a little before with some of the

0:22 ideas there. And I've got to say most the other companies have just copied their ideas from the anthropic offering. So Google came out in May with their anti-gravity agent, but really followed

0:33 around the same time with their Gemini managed agents and then June brought AWS to the game with agent core. Microsoft also June foundry hosted agents and then Langchain came along in mid August with

0:47 a managed deep agents public beta. And supposedly on September 29th, OpenAI's dev day, they're going to roll out their version of managed agents as well. So this is often on top of other SDKs and

1:01 other offerings that the companies are actually offering. So these managed agents are the providers running the harness and a sandbox, not just the model. And what I want to do in this

1:11 video is open up one of these and show you actually what's in the box. And then we'll look at why these six companies, all with different businesses, all decided that this was the product they

1:22 needed to get into. And it kind of tells you a lot about how they're thinking about this entire business strategy. Okay, so what actually is a managed agent? You can think of this as three

1:32 layers. The bottom layer is the model and of course that started out with Opus and Claude at the time. Now has moved to Fable on the anthropic side and the latest GPT models on AWS and Azure. Oh

1:46 yeah, and then don't forget Gemini. But on top of the model, the middle part is the harness. And this basically is the loop. This is what calls the models, what picks the tools, what's runs them,

1:58 and it feeds all the results back in and manages that context window. And then finally on top of these is the actual runtime. So where does the loop actually run? The sandbox that holds your

2:10 credentials that often does observability or tracing. The thing that survives and lives in the cloud by itself. So the whole managed element in managed agents here means the provider

2:21 owns level two and level three of this structure. And of course if you're anthropic or open AI, you actually own that model as well. So the pitch has been quite simple. People have talked

2:31 about the different APIs for messages, for chats, for completions. They're simply what allows you to communicate with the model. The managed agents is for all the longunning async stuff. So

2:44 you can kind of think of this as two flavors. Flavor one, they've got the loop. You send in your tasks and a config. You never have to write that loop. You never have to actually call

2:53 that model. And that's basically Claude managed agents, Gemini agents, agent core harness, and even the lang chain manage deep agents. Flavor 2 on the other hand, you actually bring the loop.

3:05 So perhaps this is a loop that you've written yourself with langraph or with ADK or it's just your own code from scratch. You put it in a container, they host it for you with some kind of per

3:15 session isolation, but they also provide the runtime for things like tool execution, memory, and other server side tools as well. So that's basically things like the foundry hosted agents.

3:26 Agent runtime used to be called agent engine a few weeks ago and may change its name again in a few weeks time. And even Langchain has lang deployment for your own langraph code. It's kind of

3:37 worth noting that both Google and Langchain are kind of having an eachway bet on both of these, which certainly for Google makes a lot of sense because you've got some things being built by

3:47 Google Deep Mind and other things being built by Google Cloud. Another access that you should be looking at for some of these things too is for the harness itself. Is it closed like Claude,

3:57 anti-gravity, agent core, etc. or is it something that's actually open- source like deep agents? All right. So since Claude managed agents seems to be the template that started this all out,

4:08 let's look at the actual concepts in there. So you've got four things here. Agent, environment, session, and events. So the agent obviously is the model, the system prompt, tools, the ability to

4:20 call MCP servers, skills. And these are things that you create once and then reference by ID. The second thing is the actual environment. So this is where it all runs. The cloud sandbox of where

4:33 it's run could either be an anthropic cloud sandbox or they are supporting a variety of different other sandboxes including some self-hosted sandboxes on your own infra. Next up you have the

4:44 session. This is one running instance doing one task. And then you have the actual events, the messages that go back and forth, the user turns, the tool results, the status updates. Now, pretty

4:56 much all of this maps over to the Gemini managed agents where they're giving you a fresh Linux sandbox as a remote environment. And one of the cool things that they've got there is that you can

5:05 actually customize that file system and mount your own skills or your own agents MD file. So, the managed agents are literally taking care of all these sessions and events and memory and tools

5:18 and skills and credentials and tracing and scheduling all for you. So why are they all doing this? Well, basically it's the economics. And if you think about it, the economics of one run of a

5:33 chat call is just literally a round trip. You call the model, you get something back, maybe your agent uses it, etc. And a good example of this is even Google's own docs say that a single

5:44 anti-gravity agent interaction typically burns somewhere between 100k and 3 million tokens. And that basically all happens because a single interaction can trigger multiple reasoning and tool

5:56 loops. You don't really have control over that. That's all running server side. Now, currently during preview, Google's not actually charging you any extra for the session server, but that

6:07 certainly hints that that's going to come just like Anthropic with their Claude managed agents. That's charging you a session hour fee on top of the tokens. So, you're now paying for time,

6:19 not just tokens. And you got to think these longunning agents like this are the highest token product a model company can sell. It runs while you're asleep. It manages all the different

6:30 turns itself. You just have to tell it what to do. Hope that it actually does that because most of these agents don't have any sort of built-in verification system and then you just have to pay the

6:40 bill. Of course, the other thing is with the companies that actually are using their own model, this all becomes a locking game to lock you into their suite of models. You want to use Claude

6:51 managed agents, you're going to have to use Claude Fable, Opus, Sonnet, Haiku. You want to use Gemini managed agents, you're going to have to use one of the Gemini models. Now, if you look at the

7:02 cloud companies, whether that's AWS, Azure, or even Google Cloud, they're taking the opposite bet. their managed agents, whether that's agent core, whether that's ADK, etc. They're making

7:15 the model interchangeable when the runtime isn't. So, of course, you're running it on their serving architecture. You're going to have to probably use it on their memory system

7:23 or a memory system that's running in their cloud. And then you got Lang Chain with the Jeep agents. So, they are really a framework company. So, they're neither a model nor a cloud. They're

7:33 basically trying to be the control plane. And the cool thing there is that they allow you to use any model, any sandbox. This can be self-hostable if you want it to be, but their locking is

7:43 going to be operational. Their goal is to get you to use Langmith. And once your traces, eval deployments, etc. are all in Langmith. Moving from that is going to be a really big challenge. So,

7:54 let me make the point. None of these services is inherently bad. They all have things going for them. Running Claude managed agents is an easy way for you to basically get something up, get

8:05 it hosted, get it running, give it a sandbox, give it a memory, give it access to all the different things that you get with the Claude agents SDK. And in many ways, if you're already using a

8:17 lot of the Clawude models, this can often just end up making your life a whole lot easier. You might want to be a little bit more careful with the cloud providers though because they're often

8:26 taking a number of different products that they had going before and are now bolting them together to make their version of managed agents. So, how should you think about whether you want

8:35 to go with a managed agent system or whether you want to take a harness and run it yourself? Well, there are five key questions that you should be asking yourself. First off, do you need state

8:46 that survives a disconnect and can resume days later? If the answer is yes, there you probably want to go with the managed agents. It just makes things a lot easier. Second, do you need a

8:57 sandbox for code executions that's going to live server side? Again, if the answer is yes, you probably want to give managed agents a good look. But what if you're designing your own agent? Or what

9:08 if your loop is actually your product? In cases like that, you definitely want to look at self-hosting and perhaps just using parts of these managed solutions like a sandbox in the cloud or a memory

9:19 tool or an observability tool. Lastly, one of the key things you want to make sure that you ask yourself too is can your data actually leave your infra and do you actually need zero day retention?

9:29 If your data is not allowed to leave your own infra or you do need zero data retention, that rules out things like claude managed agents. It's just not eligible for zero data retention because

9:39 session store history and sandbox state are both server side and that rules out a lot of enterprises that are actually looking at doing this kind of thing. So each of these managed agents both has

9:50 their pluses and their minuses. They each have their own amount of control that you actually get to run your agent in their cloud. And of course I haven't even touched on local AI. If you're

10:02 actually using a locally hosted model, you want to be able to put all of this on your own machine anyway. And then perhaps you want to look at things like Docker sandboxes, which allow you to

10:11 have a safe and secure sandbox running fully locally. So over the next few weeks, I'm planning to have a look at a number of these different managed agent solutions, different harness solutions,

10:21 both the ones that I've mentioned here and perhaps the OpenAI one if they ship at the end of September, and also some of the open agent systems that we're now seeing start to evolve out there. So let

10:32 me know in the comments if you've tried these out, what you actually found to be the pluses or minuses of the managed agents, and which ones seem most interesting to you. Anyway, as always,

10:42 if you found the video interesting, please click like and subscribe, and I will talk to you in the next video.