The OpenSource Managed Agents

0:00 Okay, so vendor neutral MIT licensed on your own hardware without a runtime bill. These are all things that Claude managed agents doesn't have. Also, the new OpenAI agent and API or Gemini

0:15 agents doesn't have them. But all of these things are the things that the agent system I'm going to show you today actually has. Today I want to talk about a way to run managed agents fully

0:26 locally or in your own private cloud with any model that you want, adding in any tools that you want, skills that you want, MCPs that you want, and being able to do it all really easy without you

0:39 having to go into low-level code and tinker with every tool call or every skill that you add. Today, I want to talk about a self-managed agent execution harness called True Forge.

0:50 Now, before we get into it, this video is sponsored by True Foundry, the creators of True Forge, but everything I'm showing you today is in an open-source repo. There's no paid tier

1:01 in anything that you'll see. And the ask from them is basically that if you find this useful and interesting and you would like to see them work on it more, just go and start the repo. That's it. I

1:12 wor with True Foundry earlier on in the year and made a video about their gateway platform and it seems to me that as they're developing things for enterprise, they're also looking at how

1:22 they can make some of these cool tools available to the open-source community as well. In this video, I want to try and untangle that a little bit. I want to talk about how you should be thinking

1:31 about the different kinds of harnesses that are out there and how you can be using them for your different kinds of projects. So let's start with the word because I think this is where a lot of

1:41 the confusion is. When most people hear the words agent harness right now they tend to think of coding harnesses. That's things like claude codeex pi om open code a whole bunch of these and

1:55 those are harnesses right they wrap a model they give it a file system a shell some tools and they run the loop for you but they're really built for one job right which is writing code. The second

2:06 kind of harness that's out there at the moment is what I would call the assemble itself framework. So this is Langraph, deep agents perhaps agent SDK or ADK and these give you all the pieces and you

2:19 write a loop and you wire up everything yourself, right? You decide how the state gets persisted, how approvals work and so on. And I've done quite a number of videos on those over the years, but

2:30 nowadays can be actually really tedious to use a lot of those frameworks if you're trying to build your own harness. The third kind and the one that's most recent is what I actually just made a

2:39 video about recently, which is the whole idea of a hosted runtime harness. So, this is what people are calling managed agents. So, you've got Claude managed agents, you've got Gemini managed

2:51 agents, and I made a whole video about this talking about the different kinds of managed agents that are out there. But the whole idea with those is that you define an agent, give it a model,

3:02 almost certainly it's going to have to be the provider's model. You'll have to give it tools, which again will be often provided for you, and a system prompt. And then that company's going to run the

3:13 loop. So session, tool calls, sandboxes, approval, all of that is handled. You just hit an API. And that third category can be really nice to use, right? I've built with it, etc. But here's the

3:26 thing. Up until pretty recently, that category really meant you only had one choice in that it has to be hosted with the actual provider. And perhaps even worse than that, you're often just tied

3:37 down to certain models that may just be not a great fit for what you're trying to do. So, True Forge is an open-source version of that third kind. And the readme puts it really nicely. Building

3:51 an agent is easy, but running one well is not. The running it well part is all about streaming, session persistence, tool servers, sandboxes, approvals, a UI. That's all the stuff that really

4:04 honestly most people don't want to write. And it's the stuff that managed agents have been charging you a premium for. True Forge is that layer as a repo that's fully open- source, MIT licensed

4:18 that you can clone and then run with any particular model that you want to. All right. So the question should be why does it matter what kind of managed agent I'm using? And this is the bit

4:28 that I think many people even myself at times have gotten wrong that you tend to think of the cost of all of this as just being related to which model that you're using and how much you're paying per

4:40 token. And then often you don't think about the harness and all the plumbing that is actually going on around this. But when you start running managed agents at volume, you start to realize

4:52 that actually that harness really is often the key thing because it's that harness that is actually deciding how many tools calls you make, what tool calls you make, in what order, how it's

5:05 going to actually decide what goes in the context window. Remember, really what we're paying for is whatever's in the context window. So, if you've got lots of tool calls in there, if you've

5:15 got tool bloat, right, where you've just got so many tool definitions and stuff in there, and you're not using progressive disclosure and things like that, you're going to run up a much

5:25 bigger bill than you would with a very efficient harness. And this is where the True Foundry team actually put out a benchmark that shows all of this. They took 14 tasks from DevRev's enterprise

5:39 bench, the kind where the agent has to join data across a CRM, an issue tracker, a document store, multiple MCP servers, all with the same prompt. And they ran three trials each and used a

5:52 blind LLM judge. And here's the thing, they didn't even go for cheaper models. They actually ran it on Opus through three different harnesses. And you can see from this diagram here, they

6:03 basically showed with the actual true forge harness that even using the same model, they were able to get the cost down by 30% with the same amount of accuracy. And that's because Claude

6:15 managed agents was using 10 million tokens per run on average versus only 3.8 million tokens on the true forge. And of course, when they moved that to a model like GLM 5.2, suddenly the model

6:28 cost came down. the whole thing now was down 75% compared to the original claude managed agents version. So even if we skip the GLM 5.2 and just look at the same model, same answers for roughly a

6:43 third of the tokens, that was all harness decided. So the fact that True Forge is actually open source means that you can pick any model that you want here. You can decide if you want to go

6:55 for a more expensive clawed model or open AI model, etc. But you can also go for much cheaper open models like I'm going to show you in here. So, if you just take one thing away from this

7:07 video, it's this. The harness, not the model, is usually what's driving your agent costs. And the harness is the part that you've actually got a lot of control over, even if you're not running

7:18 your own models with your own GPUs, etc. Now for me I would say while the cost savings here are the headline for me the lockin thing is probably a bigger deal than a lot of people are watching just

7:30 in the park sweep we've seen a number of different news items about what user data these people are actually training on and how they're actually using user data and you've got things like data

7:41 retention if you want to have zeroday data retention you cannot use clawed managed agents if your agent definition lives inside a hosted runtime then and the system prompt, the tool wiring, the

7:53 approval rules, the session history, all of that sits on someone else's product. And perhaps that's fine up until the day you want to try a different model or you want to try a different sandbox or you

8:06 want to try a different configuration, etc. You don't want to get locked into these services where you become dependent on them for a lot of things beyond the model. We're already seeing

8:17 open models get so good that really high levels of intelligence are now being democratized so you can access it over lots of different providers. Even if you have objections or legal things around

8:29 sending your data to certain countries, the cool thing now is with open weights models, there are providers popping up everywhere. They're running the same weights with the same API endpoints. And

8:40 I hear this a lot from people building inside larger companies. They often have a mandate to run things on prem or at least inside their own cloud account and a hosted agent system just isn't going

8:50 to get through procurement through compliance no matter how good it is. So they fall back into a category of trying to write the entire loop themselves and that can also become both an inefficient

9:02 and expensive option. And then of course if you do have some local hardware that you can run some local models on we're already seeing the smaller models get better and better and this whole system

9:13 allows you to integrate with things like Olama with things like LM studio to run models locally. All right so True Forge is a MIT licensed TypeScript repo from the True Foundry team with one MPX

9:27 command. You can basically spin this up. It takes care of a local SQL light database. all your data is staying locally and it's running on a local host. There are also hosted modes if you

9:38 want to do it with things like Postgress or Reddus and Docker, but the whole local mode is a very deliberate step on True Foundry's part to make this as easy as possible for you to get started

9:48 trying it out. Now, they have a whole chat UI, but don't be fooled into thinking that this is another chatbot. It's not. The chat interface there is the test bench. It's where you poke

9:58 around and play with the agent and test things out before you actually save it. The actual main thing here is the runtime and it's actually exposed a few different ways. You've got the UI that

10:09 you can start off with, but you've also got an HTT API endpoint and then also you can use that with a TypeScript SDK. This means that you can build something in this very quickly, expose the API

10:23 endpoint, and then have other apps ping it. So, this makes it ideal if you've got people on your team, for example, that you want them to basically run different agent workflows, but you don't

10:36 want them having to worry about how to configure the tools, how to actually prompt it, right? How to do all of that. You can actually just bake this into something like Slack, a mobile app, any

10:48 sort of software that you wanted to write. So, let's jump in, have a look at how this actually is all put together. All right, so to get started here, this is pretty simple. We're just going to go

10:59 to the readme docs and see go to the quick start and then I'm going to just install this locally to get started. And you can see this this will basically install all the components in here.

11:10 everything from being able to deal with sandboxes, MCPs, skills, all those sorts of things and it's actually now running locally so that I can get started. Okay, so once you're in the actual interface

11:22 in here, you can come into the settings and you can set up models, connectors, skills, and sandboxes. So models here, you've got your standard sort of offthe-shelf models. If you do want to

11:35 use Claude, you can use Claude. If you want to use Gemini, you can use Gemini, etc. You've also got a bunch of open providers or you can configure something totally private. Here I've configured up

11:46 my own API endpoint to run DeepSeek Flash 4.1. So this model is just out. The previous four Flash was one of my favorite models. So I think like a lot of people I'm quite excited to see how

11:57 this new model goes. But you could configure lots of different models in here. Next up you've got connectors. So you can see I've already gone and configured Tavilli which is a really

12:07 good search tool. You can get a free API call. I think you get a thousand calls a month or something for free. Also, Deep Wiki is one of the ones that I like, but there are lots of others in here. So, if

12:17 I wanted to add my notion, I can go ahead and do that. If I wanted to add bright data for doing any sort of scraping or things like that, I've got that in here. Or I can just come in here

12:27 and totally configure up a personal MCP server. So, that could be running locally, that could be running in the cloud, etc. And then you've got skills. Now skills is obviously one of the best

12:37 ways to actually get started. So I'm going to put in the Tavi research skill there. So I'm going to enable that so that the folder contains the skill.md. I can then just import that and use that

12:49 as well. And then lastly, we've got sandbox providers. So they're a bit light on this at the moment. I think True Foundryy's also got some interesting things around sandboxes that

12:57 you might see coming in the near future. But once I've got this set up, you can see I can just come along and I can do a new chat. Now the chat ID here is really just to allow you to test that

13:10 everything's connected up and stuff like that. So you can see I've got my DeepSeek VFL.1 flash model in there. I've put on to Villi as a tool in there. And if I

13:21 wanted to import some files or something like that, I can use this just like a normal chat interface. But it's very important that you understand that this is not the product, right? This is just

13:31 to basically test the product. So the idea here is that okay, now I can see that okay, everything looks like it's running. I'm getting reasoning tokens back from 4.1 flash. I can see that it's

13:45 running through the different tests and we've got the ability now to run and see what actually is going on here. You can see that the agent has gone off and just me saying that this is basically a test

13:56 to wire things up. I actually was just planning on testing the actual LLM, but it's gone through the whole system. And this is the cool thing of this, right? All of these things are now integrated

14:06 as an agent. So, it's acting as an agent with these tool steps and with this thinking. So, it's taken this as like my task that hey, we want to check that everything's set up. It's gone through.

14:19 It says right, yep, we've got the date time, we've got the MCP setup, we've got our tool execution, everything on that is working. We can see it failed on a sandbox because I haven't set up a

14:30 sandbox yet, right? So, it's good to know that this is already starting to work. Okay, so if I want to build a new agent, let's say I want to build an agent that's going to go and get AI news

14:43 for me. So, I'm just going to do a very quick one here just to sort of show you how you would do it. Okay. So, you can see I've got my prompt there. And what I'm doing here is basically just going

14:54 through this, letting it work out the different things to do. And the cool thing is it's doing all this in the background. It's using Tilly. I didn't need to tell it to do that. It basically

15:04 worked out to do that. It's also then gone to the VengeB site, which is the link that I gave it. Worked out that okay, we probably want the category A in there. Now I don't have a sandbox set up

15:15 so it's obviously working out that that's going to be an issue but you can see very quickly it's basically worked out itself what it needs to do like extracting the homepage trying out

15:24 different things etc for doing this and then the cool thing is it's then used to villi to actually do searches passing in info knowing that I'm just looking for the last 48 hours we can see what it's

15:37 actually done there with Tavilli and you can see that finally it goes through it finds out like the Deepseek v 4.1 flash release different releases that have come up chat GPT for financial services

15:49 etc. And then even for other ones it's then focused on checking the date right to see if those are right now once we've got that we can then come over here to actually save this I then basically put

16:01 in my instructions which is like my system prompt here for doing this. I can add any skills that I want. So I'm going to put in the actual Tilly research skill in there. And then I've got to

16:13 decide some things like, okay, do I want the agent to ask clarifying questions? Do I want it to have dynamic sub agents? Do I want it to have basically like return a generative UI? Actually, I

16:24 don't want the generative UI or the clarifying questions on this. I just want to sort of ping it and basically have it return the result. So now I can save the changes and it will go through

16:36 and create this agent for me. And you can see now if I want to basically just try this out, I can come in here and put in the prompt to kick it off. And you can see sure enough it's going off. Even

16:48 though I wrote just get me the news, it understands that this is going to be AI news. I actually changed it to be 72 hours in the system prompt there. It's got today's date. It's got the different

16:59 parts that it's going to basically get there. And you can see that already it's got a bunch of sort of the recent stories that have come up. And so now it's going to basically check the dates

17:09 of those. And all of this is running with the deepse model, right? So this is a very cheap model to run. And you can see that its final output is coming back with the actual stories with a summary

17:22 and with the the source URL for each of these. So this is pretty good, right? If I check something like this, sure enough, this was written in the past day or so. And then it's also put in some

17:32 honorable mentions there. Okay. So if I want to use this with Python, I can actually just come in here and basically set up a a little UV project with just a single Python file. If we look at this,

17:44 it's just going to basically call the local host version of this. This is where the agent is currently being hosted. You could actually have it running in the cloud if you wanted to do

17:55 that. Obviously, I think going forward you'll see True Foundry have their own sort of managed versions of this which lets you prototype everything locally and then ship it to their cloud to run

18:05 regularly at scale kind of thing. But you can see if I want to basically run this now, all I have to do is UV run main.py. It will basically set it up. It will go through setting up the different

18:18 tools, etc. And then it will go off and do the task. And here I'm just streaming all the things back. But if you wanted to, you could get it to basically just give you an output file or to save to a

18:29 database or something like that at the end. And you can see now that it's finished off here. We've got all the results back. We've got the Open AI launches financial services. We've got

18:40 Deepseek V4.1 Flash in there. And for each of these, we've got the Venturebe article. We've got the date. We've got the different information in there. So just to finish up, I would say this is

18:52 definitely worth checking out if you wanted to run agents locally on your own machine, etc. And the cool thing is once you've actually created these agents, they're stored there. So you could

19:03 actually install this in the cloud and import them in there and use that. Now looking at their road map, it looks like they've got a bunch of things coming up. You can see that they've planned to have

19:11 a gateway integration, which totally makes sense. True Foundry has a lot of sort of gateway features both for models, for MCPs, for locking things down like that. It looks like they're

19:21 addressing probably my biggest issue with it is that I would like them to have a local sandbox execution and it looks like that's coming. It'd be great if you could also lock that down with

19:30 Docker sandboxes or something like it just to make it fully safe and secure. And looking at the rest of the road map, it all seems very sensible stuff if you wanted to do this kind of thing. And a

19:41 lot of it is perhaps stuff that is in the lockdown versions of managed agent. So it's great to see that this is all coming in the near future. So go give it a star on GitHub. The links are in the

19:51 description. Really this is giving you all the advantages of a managed solution without the vendor lockin and without the downsides etc. So let me know in the comments what are the biggest features

20:02 you would like to see added to this. Talking to the team at True Foundry, they do seem very committed to making this work, both for their customers who are using it, but also for the general

20:14 open-source community. Anyway, as always, if you found the video useful, please click like and subscribe, and I will talk to you in the next video. Bye for now.