Ex-NASA dev reveals his Agentic Engineering Workflow

Stop playing with your coding agents and get back to work. They can solve problems really, really well, but they're not going to be able to write maintainable code without you in the loop helping. The thing that I think most people skip is actually what I call like program design. It's like before you let the agent go cook, there are decisions that it's going to make that you might not like. >> If you can tell it a measurable output, the agent will move mountains for you. >> You can tell the model how you want it done, and if you don't tell it how you want it done, okay, it might do it in a terrible way, but it also might figure out like a better way to do it that you never would have come up with. >> This is Dexter, also known as Dex. He's the one who invented the term context engineering, and he's also known for creating a software factory where AI agents ran for four months straight without any human input. In this podcast, we talk about why benchmarks don't matter, his unique program design system, and why pull requests are dead. If you want to be in the top 1% of agentic engineers, watch until the end. This is the David Andre podcast. Enjoy. >> You had a thought on benchmarks not being an accurate way of representing agent's power. Can you tell more about that? >> basically about the idea that like a lot of benchmarks are like one-off problems. They're quick, one-shot, fix the tests, do all this stuff, and like, I don't know, I am not a machine learning expert, but like I was doing a bunch of research on this cuz I wanted to match my vibe understanding with like how this actually works in the real world. >> let basically agents run by themselves. Give us the backstory. >> So, like software factory's been around since 1968, but like I like to think of like the 2022 software factory, right? Which is basically like right before AI, you had a team of humans, and they would put stuff on a list of things to do. This was your Jira, your linear, your state machine, your issue tracker, and someone would go build the thing, and then like eventually you would do a pull request, and you'd do your CI/CD, and all kinds of We've automated a ton of stuff, but we still review the code by hand. If something is wrong, we loop it back to building the thing. Eventually it goes into prod, hits our users, they complain about stuff, they ask for features, and then we go around the loop over and over again. And then eventually we put on monitoring so we can wake people up at 3:00 in the morning because apparently we love to wake up engineers up at 3:00 in the morning when things break. But this is your like software factory before AI. Basically the first thing that happens after that is you replace like person builds the thing with agent builds the thing. I'm not going to get too much into like orchestration and harness and sandbox and model. Like you need all these things, but basically like every single thing that comes off your list, this is the ramp, the work OS, the Brax, they all have their the Stripe Minions, they all have their like system that does this. And this is the first thing you replace. And you probably leave most of this other stuff in here. And basically like now this part takes minutes or hours, and this part still takes hours or days. I guess like before we had agentic factories, we basically had like people noticed that like building stuff took hours or days and reviewing stuff took hours or days. And so we did this stuff to like get more leverage, which is basically like we'd sit in a freaking meeting and talk to other engineers about what we were going to do so that like instead of spending 6 hours reviewing it, maybe there's like a a better chance that like oh yeah, this is what we all talked about, the PR matches what we explained, and there's like very little changes required. And so like we want to basically optimize spend an hour up front to save hours in the review time, in the build time, make sure we're not building the wrong thing, etc. This department is now fast, but this part is still slow. And we bring in like agentic code review and it catches all the small stuff. And we let the agents like just poke at the app and test it with browsers testing and stuff like this. And that makes this faster, but like if you want to read like the code that matters or any of the code or all of the code, whatever it is, like this is still your bottleneck of like how do you build trust that the thing that was built is going to work today and it's not going to like slow you down in a month, right? >> If you're, you know, getting tens of thousands of lines, you cannot review that manually. >> Yeah. >> That is that is the bottleneck for sure. >> So you do what you can to build trust. You say, "Okay, well, we threw Codex and Opus at it and they both said like they both found 10 things and we fixed all 10 of those things. So, now I have higher trust that this is going to be like I'm not going to get a ton of complaints and feature requests, and I'm not going to like get woken up at 3:00 a.m. cuz something broke. And then we can do some stuff like routing incidents straight into the factory. So, when the alarm goes off at 3:00 a.m., I don't wake up to an alert, I wake up to a pull request. And we can do the same thing with like user feature requests, right? Instead of like going through a support queue, we triage some of the support queue straight to the agent and it just goes and fixes the bug. Have you done any of this yet? >> Yeah, I'm implementing this as as much as I can, to be fair. For example, for my current thing, Deep API, I'm I have like GLM 5.2 to review every single like uptime incident. So, anything that's like suspicious or anything that it goes down before, you know, I have to like read it and understand it or use AI to understand it, but now I just get like an AI report of like, "Oh, this is what happened. It's actually the provider. We cannot do anything about it." Or like, "Oh, yeah, this is caused by missing migration. Do this." >> Yeah. You're like, "Oh, Anthropic is down right now. Like, you just have to like tell your users. And like, here's a draft of a message you can put out in your Discord channel or whatever." That's cool. Where are you running that? Is that running like in a in a in a cloud sandbox somewhere? >> It's split between render.com and just cron jobs on Vercel and GitHub Actions. >> Okay. So, you have cron jobs on Vercel, and then like when incidents happen, you pull them in and then render.com runs some little agent loop that does inference and then like sends it somewhere. >> Yeah. >> Cool. >> Now, launching your app is just step one. The moment some users show up, the question becomes, "What are they clicking? Where do they drop off? What's breaking?" And right now, that information is scattered. You have one tool for analytics, one for surveys, another one for error tracking. But with PostHog, you get all of this in one clean platform. You can link all your data sources and query everything alongside your product data. So, you can find real correlations and real patterns instead of just guessing. There's also session replay, so you can watch recordings of real user sessions to see where exactly they get stuck. But honestly, you don't even have to watch this yourself. PostHog's new AI tool, replay vision, watches those recordings for you. It automatically spots any friction and an agent opens a pull request with the exact fix. So all you have to do is just review it and merge it. That's the whole idea. PostHog makes your product self-improving. And this is essential if you're building software in 2026. For example, most people have no idea how to monitor their AI inference calls. But PostHog does all of this for you. It tracks the prompt, the response, the token cost, and the latency. Everything per user. So you can see which features are burning money, which agents are misbehaving, and which prompts are actually working. This is what you need to build a great product. You can try PostHog for free by clicking the link in the description. Oh, and thank you PostHog for sponsoring this video. So this is like when people talk about the full software factory, this is like getting towards like how do we take ourselves out of the loop? If something can be just one-shotted by an agent, even if it's only like 30% of your work, only 30% of your incidents, that's huge. That's like you're getting close to doubling your productivity, right? Cuz if you can automate 50% of your work, you can now ship twice as much. And so like now the thing is like cool, now our job is more just like ask for features, evolve the platform, think about strategy, and then like how fast can you review the changes? >> Yes. >> Which gets us to this thing that I don't think works, which is the light soft factory, where we just say like okay, we're just not going to read the code. Like how can we build a system where we don't have to read the code or we have to read as little code as possible? Do you still review your your PRs that the agents write? >> Not like all of them. If there's a small change, I just ship straight into production. Like one maybe controversial opinion, and again like I have less experience than you, so like you know, I have to preface that. But like I have a good sense of like my prompt, right? If it's like a small front-end tweak, I know what the agent's going to do. Like I just have like over thousands of hours building with AI, probably over 100 hours with these new models. But like I I just know like what type of things the agent is likely to [ __ ] up and what I can trust with. If I'm not sure, again, I do the basics, right? Review it with the other frontier model and do more testing myself, stuff like that. >> That makes sense. And basically the idea is like when when you if you want to get out of code review, you you end up investing into all these other things. You make it so the agent can test its changes live. Maybe it can even send you a little video when it's done building. This is like the cursor sandbox agents do this. You make your agent to code review way better. You make your regression testing way better. Every time something fails, you go figure out what it was and you make sure that that particular failure mode never happens again. You make your CICD checks better and more robust. You add more different types of linters. complexity monitoring, all this stuff. And so now your job really is just like how much stuff can you think of? How many how many features can you ask for? How many features can your users ask for? And like you can actually run this thing where you don't ever read any code. I personally don't think this works and this was like kind of my epiphany and like me I mean, I think if you've used agents for a while, you kind of have this vibe of like if you do this for like way too long, >> You can like completely lose the touch of the code base, right? So then like the maybe the part that like I would add is really spending time on the agent educating you. Like checking like is the human still in the charge? You know, like giving you some some quizzes, giving you some like HTML visualizations and kind of slowing down on purpose when you start to lose the grip. >> Yes, exactly. >> Real quick, if you want Dexter's exact program design system that he uses, it's going to be available as the first link below the video. It's completely free. Go grab it now. >> We even had a prototype feature that I didn't end up shipping but we may bring back which was like while the agent is working on stuff, you can pull up the session and it'll quiz you on like what is the current state of the code base and how does it work today? And then like how does the new implementation work and what does it change? And you can like test your understanding with like multiple choice questions with different mermaid diagrams and stuff like this. Anyways, we we did this for a while. Basically like the idea here is like if you stop reading the code, you might and or you absolutely will eventually hit an issue that the agent can't solve. You're like, "Oh, this is broken, go fix it. Oh, this is broken, go fix it." And like you try all of your like most intense prompting and you get three of the smartest models together to do a council on it and they just keep thinking it's something that it's not and they keep shipping fixes that don't actually fix it. And so you have to go into the code base that you stopped reading 3 months ago and spend like days or weeks like one, like trudging through like sloppy code cuz you stopped reading it. And two, like trying to figure out what is actually wrong. And this has happened to us at least once or twice, which is why we kind of like threw out our We did this in like July of 2025. We had this light soft factory where we were like really zooming out and not really looking at the code so much. We reviewed the plans and we did like collaboration on like the tickets and the like what went into the to-do list, but we weren't reading the code. And then we hit a bug and like for weeks it was like, "Okay." Our site was like I mean we we were shipping a desktop app, so like the app was buggy, users were pissed, we were miserable reading through a bunch of like slop code trying to figure out what the hell happened. And it just it sucked. And I kind of have this thesis that like the odds of this happening to you if you stop reading the code are much higher than the odds of it not happening to you or like that the models will get smart enough by the time it happens that you that that they will be able to solve it. >> Maybe I will challenge you on the wording, right? Like is code the right word? Because like you can replace code by the logic. Like if you understand the underlying logic of your software and like are we using a certain API schema or like what's the difference between hashing and encryption? If you just understand these concepts, you don't necessarily need to look at the code per se. You need to know like the logic of like, "Okay, what does the first 5 minutes look like in the onboarding? What does it look like when someone tries to cancel? What does it look like when this API endpoint fires?" Maybe I will challenge the word code and like replace that with logic, no? >> Like know what the product is. I'll jump ahead to kind of like what we I'll go back to like why I think this is. We do a whole thing in like RL and how the how the cases work and how it's like kind of obvious that that that is how it happens. But we basically go through like four stages and I can demo this in a little bit with our tool, but like you don't need any tool or any specific set of skills to do this. This is just like what I think is the right way to think about aligning with AI and kind of staying in the loop as much as as possible. And the first one is like the product part. Like what user problem are we trying to solve and like >> Yeah. >> if you can do it, how do we measure it, right? Because if you can put an experiment on your website and say like, "Okay, this is hard to use." The agent can try three different things and then it can check the data every day and decide which one works best. That's incredible. Now you're really getting on autopilot, right? >> And this is something that the latest generation of agents is like doing way better. Like I noticed, you know, with Fable and 5.6 so like I I don't have to tell them to run tests. I just noticed like if they do something medium-sized or a large, they just try to do way more testing by themselves. So I think yes, in certain areas you still have to like think of ways to measure and track, but with the models getting better, they're they're doing this by themselves. >> When you say testing, are you you talking about like doing like browser testing and stuff like this? Like using something like agent browser? >> No, no, no. Like if I'm building some feature, like something like your deep research or scraping and they switch a provider, they would test cost, speed, quality, you know, using LLMs as a judge without me even saying like test it after changing it. And that didn't happen in the previous generation of models. >> That's sick. We have seen that a little bit more. They got much better at like computer use and browser use and they're much more likely to do it without being told, for sure. The whole like concept here is like back pressure, right? Is like how can you take what you build and give the model like deterministic like yeah, LLM as a judge is is fine and LLMs can review code and they can review outputs, but like if you can actually say like, "Hey, here's a number, like the conversion on my website that is directly tied to the success of my business, to how much revenue I'm getting, to how many users I'm converting." That is crazy powerful. >> I was about to say that's the same idea behind like the slash goal. If you can tell it a measurable output, the agent will move mountains for you. >> That was the same thing as auto research, right? It's like just go make this CUDA kernel faster until it like whatever it is is you've reached the 20% resource reduction or whatever it is, right? >> Yep. >> And so like here what what some of the things we like to do here is like basically I'll actually pull up a doc that we did. Yeah, so here's like a PRD I made in in Humanlayer that is like here's how you bring your custom workflows into Riptide. So it's like we have here's the problem to solve, here's how we measure success. And actually did this Amazon thing which is like write the blog post before you build the feature. Think about how you're going to explain it to users and why it's valuable before we ever write a line of code. So we're building this new JSON format and then it comes with a bunch of like we do HTML mockups to understand how like what it's going to look like. I did a bunch of iterations on this and then like basically all of the different views in the app that are relevant, we kind of just like prototype them out with like plain HTML and it it it seems to work really well. But this is like the product part. There's no tech here, there's no architecture, we're not talking about databases or schemas or anything. It's just like what is the user see? What is the user spacing stuff? >> Yes, having perfect clarity about like what do you actually want to build and why do you want to build it before getting into any prompting or coding. >> Yeah. And like people say like oh but this is just extra work and like the models are good enough now and it's like sometimes, right? You can roll the dice and just see what comes out, but if you spend 20 minutes going back and forth and figuring out what this looks like, you can do a lot. You can like raise the chance that whatever the agent ships does not need a bunch of like hand polish or tweaks or you got to like file a bunch of changes to the UI or whatever it is. >> Yeah, the models are also good enough at like doing what you tell them and never calling you out about like yo, you've been working on this for days and there is zero progress and like you still have zero users. So like you know, that's the dangerous part. >> So I mean you mentioned like understanding the product part, that's one example. The next thing we do is like system architecture, right? If you want to save time during the review, you pick whoever would review the PR. I mean, if you're solo, then this is a different story, but it's like whoever would review the PR, if you're especially if you're working on a team. We focus a lot on like we work with publicly traded companies, a lot of like series B plus fintech companies. People where you can't just like vibe it and hope it works, right? And like, "Oh, if it's broken, like we'll just fix it tomorrow." cuz it's like in those industries, if it's broken, you might get fined a million dollars if you do something wrong. And so, they are reviewing the code and stuff. And so, we can do this high-level architecture thing, right? Of like, "Hey, here's how the Here's how the services are going to fit together and here's the flow. Here's the new endpoints we're going to make. Here's the new tables and then like outlines of the queries we're going to do." And this is like going down one more level, right? This is what a lot of people do. A lot of people have gotten to the point where they're quite comfortable working back and forth with models to design the system architecture. And this is really good. Do you do stuff like this? >> I'm not at that level, so, you know, right now I'm launching a new startup with like two developers, so I'm not there yet. >> So, yeah. The The other like asterisks on this, right, is like there are times to vibe code. This is like less about vibe coding than anything else. Like, if you are a pre-product-market-fit startup and you have to figure out what's worth building and you just have to like put as much different things in front of customers and figure out what they're willing to pay for. Like, this is probably overkill. But if you're on any kind of team, even like around like five plus engineers where it's like, "Okay, we know what we're building and we have people paying for it and we need to make sure that it like is going to be around in six months and that we're going to be able to maintain it." This is kind of step one. The thing that I think most people skip is actually what I call like program design. It's like actually going before you let the agent go cook, there are decisions that it's going to make that you might not like. We see a lot of people actually going one layer down of like, "Here's actually what the call stack of the program's going to look like." Actually, Dylan Mulroy from Cloudflare did a tweet about this recently that's really good, which is like most of his plans actually end with like, "Here's what the tests are going to look like and here's what the call stack is going to look like when we go build this." And again, the most important thing we talk about is like I use the word leverage a lot. How can you spend a little bit of time up front to increase the odds that the thing that comes out is not going to need a lot of fixes and changes before you can merge it. Or after you merge it, you're going to have to go back and clean it up, right? >> This is basically like the inverse process. I don't know if you've seen a tweet by Victor Tali. But basically, it's like when the models do a change, ask them, "While working on this, which choices did you make that you're not confident of?" >> Yeah, I've seen this. Yeah. >> Yeah, so that's basically doing this before the the model runs. >> Exactly. Yeah. My take is basically like once the model has written thousands of lines of code or even hundreds of lines of code, it is harder to change then because you're deep in a context window. You've done Even if you use a lot of sub-agents, like there's a lot of context there and you're already kind of biased in one direction by what the model chose in the first place. The sessions that generate these kind of docs are very context light. Like the model has all the understanding, but it's doing it in a very token efficient way. I have one of these in progress. Um let me go pull it up. This is like the conversation back and forth. We're at 43,000 tokens and we've already made uh a lot of decisions. We read the PRD in this session and then we've already made a lot of decisions on like how this is going to fit together or what the endpoints going to look like, like what's the flow, what's the actual like This is this is our program design. Actually, this is just the architecture still. I haven't even gotten into the program design. But the point is like you can make a lot of these decisions in a very context cheap way, which means you're getting the most model intelligence, right? The deeper you are in the context window, the long context window, right? So, that's that's like kind of the program design side of it. We look at like actually, "Hey, where are we going to put these files?" I'm sure people see like, "Wait, why did you put that over there?" It's like, "Okay, this is one question that you can go back and forth with on the model." And we use these like code blocks a lot. We prompt the model to use these things that are like very easy for humans to quickly read and decide is this right or is it wrong. We just define the types and the method signatures, but not necessarily like getting into the implementation details. And the last one we do is like I recommend people do is what I call like vertical slices. Matt Pocock talks about this all the time. We I talk I actually did a YouTube with him in like January. We talked about like tracer bullets. Um and the basic idea is like models love to build things in a horizontal way. They like to do everything at one part of the code base in each phase. And so like there's nothing along the way here that you can test. Basically is the challenge. Is like you do the database and the services and the API and front end. And like at the end of it you're sitting at like the other side of thousands of lines of code and there was nothing to check along the way. So again, coming back to our like once the code is already written, it's more work to re-steer and it's harder to kind of like pick it up and change it in a big way. And so what we always recommend is this thing we call vertical slices, which is like you have these layers of your system, but like when I built code before AI, I would always build like a mock API endpoint first. And then [clears throat] I would And then I would stub out the front end. And then I would get the front end right. And then I would wire it down. And then I would do a migration. And then I would do all my business logic. And then I would do all my error handling. I have never seen a model do this without human getting in the loop and like telling it what order to do the things. But this means that if you want to, you can test it along the way either manually or with curl or with browser agents or whatever it is. It's like get it working all the way end to end first and then go add logic and then go add business cases. Does that make sense? >> Yeah. Okay, so you say that I I realize like I've never seen a model be like, "Okay, I'm just going to build a empty version of this end API endpoint, test if it works, then add 50 lines, test if it works." It's just always like, "Okay, so this is going to require complete front end feature plus 500 on the front end. Then we're going to do plus 400." Right? So like as you're saying that I recall like this you know, before I built some stuff before LLMs, not as much as you, but like you know, when I was like 15, 16 and I was building like mobile games, that's how I did it. I was just like, "Can I get something to work?" Okay, it works. Let's make it slightly more complex. Okay, it still works. And then like adding one step at a time. >> This is like how you learn a new programming language, right? You do the Hello World, and then you change the Hello World until it's kind of looking like the thing that you want. And you like push the boundaries of the language, and you understand the shape of the problem as you're going. And like >> Yes. >> I think this is a great way to save a ton of time, especially if you like you're going to end up reading all the code at the end. Like you might as well check these first two steps and like go check peek at the code as well and be like, "Oh, I don't like the trajectory this is on. Let me re-steer it now when it's cheap versus going all the way to the end and then like hope that it's right. And if it's not, it's much more work to change it." >> Yeah, I mean it's the same in business, right? Like your first customer, deliver them and do everything. You don't like spend like, "Okay, let's spend 10 months designing the perfect Facebook ad campaign so then we're ready for $10 million a month." It's like nobody works like that. >> And so this is all predicated on this like thesis of like, "Hey, you're going to have to read the code for now." They can solve problems really, really well, but they're not going to be able to write maintainable code without you in the loop helping. I had this vibe, and lots of people I talked to had this vibe, and I wanted to go figure out why. So I like dug in a lot into like how reinforcement learning actually works, and like >> Okay. >> why they can't get the models to do this. I'm sure they're trying and they're working on it, and it may be solved soon, but like one of the things I focus on a lot is like, "Okay, if you want to solve problems today, and you don't want to just sit around and YOLO prompts until GPT-7 comes out and hope that that's going to be better, then like you should understand one level down on like why they're bad at the things they're bad at so that you can have an intuition about like where to get involved versus like you said earlier, it's like there's certain things where I just know the model's going to do it well and I don't need to be in the loop." So I had this like visualization inspired by my buddy Calvin French-Owen who used to work on Codex. So we basically give the model a problem, we have it try to solve it a bunch of times, we put the temperature high so it tries lots of different stuff. >> Yeah. >> And then we score each of these traces. Like did it actually solve it? Was the code concise? Is it verified from the outside, and then we reinforce, right? We take the traces that did well and we give them make them more likely to happen by shifting the weights, we take the traces that did badly, and we make them less likely to happen. And the thing is is like if you look at the problems, basically there's no penalty for bad design here. All these benchmarks, like you can look at Sweep Bench multilingual, you can actually look at every single problem in here. Somewhere in here we have Yeah, so here's like the golden patch for this problem. So the actual like correct answer for this, they're like short little changes, maybe like 100, 200 lines, and then we have like what are the new tests that need to pass after the model solves it. So it doesn't have to solve it in the way the human solved it, but it does have to make the test pass that the human wrote, which is a little bit shaky sometimes because it means that like depending on how the model implements it, it may be correct but the test don't pass. But this is really simple like problem from Fast Language Ruby project, which is like, "Hey, if you call this thing in a certain way, you get this like null pointer exception and the stack blows up." And the the human fix was like just to basically if it's nil, put it in empty list. And we have these tests basically is like, "Okay, after the model's done writing the code, uh this is the test that should pass." This is the size and scope of some of these problems. And so basically like the way we grade this is we have the we have the problem statement and we check out the code base at whatever commit it was at before a human solved this open source problem on the web maybe 10 years ago. The agent tries to solve it, it builds a patch, and then we like grade it in a sandbox. And so we see like, "Okay, did you We still got whatever changes the agent made to the test because I'm sure you've seen agents like basically comments out test to get them to pass." And then we put in our tests and then we see do they pass. And like, "Okay, did we fix the Did we fix the new problem and did we do it without like breaking any of the old tests?" Then you get a point. >> So in this case you don't care if that specific like open source, you know, solution is in the training data. >> So this is one of the problems. I'll show you what some of the newer benchmarks are doing. One of the things they do is also like trying to make sure that the the benchmark set is really held out because yeah, the models have read a lot of code. It is possible that during training the models have actually seen this code, this pull request, this repo already. That's like a totally orthogonal problem to like this problem does not penalize the model for writing slop. It just doesn't. And so like if a model gets 99% on SweBench, I don't give a [ __ ] because I know that it's it's not being penalized for slop. So like yes, it's better at problem-solving but it's not going to convince me that I can stop reading the code anytime soon. >> Yeah, if it if it solves it with like writing tens of thousands, hundreds of thousands of lines of code, you definitely don't want that in a real business, you know? So even though technically it solved that problem, it's like you would probably fire that engineer. >> Yeah, exactly. And this is why we get like patterns like this where it's like JSON.parse and then it's like okay, we catch the error and then we log it and then we re-throw it or return nothing and it's like this is completely unnecessary. It makes your code worse. Like we get these weird type casts everywhere where it's like it's just just trying to get the test to pass, right? And I think like basically the reason why we don't verify quality is cuz it's really, really hard. In reinforcement learning, you need what we call an oracle, right? A thing that can tell you like verify whether the solution was correct or not. And maintainability has no fast oracle, right? You can run the test in a couple seconds and you can run millions of RL loops on on on thousands of problems and you can do it kind of efficiently, but like basically the cost function of bad architecture is measured in weeks and months. And so if you make a bad decision over here and then you keep shipping and you do random stuff and then suddenly get an incident due to something that happened over here cuz it just kept getting more like spaghetti and unreadable. Like there's no it's really hard to like back propogate this during a training run to like make the model not do that again. And the other problem I have with all the benchmarks today is like software is all about discovering problems as you go. You ship something, give it to users, you get feedback, you ship another thing. And every single benchmark, even the frontier ones, like the model knows the whole problem up front. We have no benchmark that like feeds the model like five features in a row and proves whether it's good at like making the code base not just solve the problem, but making it easier to easier to change so that when the next thing comes in it can keep working and keep working and keep making progress. >> I think the big challenge is going to be like trusting the benchmark. You know, 2023, 2024, everybody was like only looking at that, but now people are more skeptical. It really is like a big challenge to find a benchmark that doesn't get saturated and that's actually useful. >> Yeah, exactly. Like I have some ideas of benchmarks where it's like, okay, you have a you have a road map of 20 features and the model doesn't know any of the features in advance. It gets them one at a time. I think [clears throat] I would trust a benchmark like that way more than a benchmark where it's just like, hey, go clone all of Redis, write 200,000 lines of, you know, Rust code that is like Redis compliant and like, oh my god, it cloned Redis. That's crazy. Like so impressive. And it's like, yeah, but are you going to be able to add a feature in two weeks or is it going to be complete chaos? There is a new one from Cognition. There's a couple other ones that I that I'll I mention in this post, but basically there's a new one from Cognition that is like better in the right direction. One of the most interesting things that they do here is like you still have a history of issues and humans create the issues. And so you take the issue and the code base checked out to that shot, the LLM goes tries to solve it. And we have a verifier, like do the new test pass? Did it not break anything? But then we have like a there's like a golden solution and we basically have an LLM that says like, okay, look at what the model wrote and look at what the human wrote and decide if they're like functionally equivalent. Or like did the model miss some nuanced detail? Like even if the test fail because they're not like shaped exactly right, did it solve it did it solve the problem? And so this is like a little bit more flexible in terms of like we don't penalize the model for writing code in a different way that the human did, which is interesting but doesn't solve the slop part problem. Uh but the really cool one is we have this LLM quality judge, right? Um which is like, okay, cool. We have a bunch of part of the data set is we have a bunch of quality rules of like, hey, if you're writing C++ you can never log like this, you have to log like this. And this LM looks at the rules in the patch and says, okay, like thumbs up, thumbs down, did it follow the rules? This is all things that like people are doing in their software factories today. The most novel thing is not like novel in terms of software engineering, but basically they look at the test the model wrote and they say, cool, let's remove the model's patch and run the test against the pre-patch code. Do the test fail there? And if they don't fail on the pre-patch code, then that's like evidence that the model wrote test that don't actually test anything. They're just like testing [ __ ] >> So basically to explain this like in a simpler way, you want to bake in the other less obvious things that like you can label as taste. You want to extract words from the word taste, which is not helpful, into like numbers and tests and, you know, measurements. >> Exactly. Yeah, it's like how can you give deterministic feedback or something a little more like LM quality judges obviously not deterministic, but if you keep it very focused and say like say yes or no, did it follow all the rules, you can get a little bit more of a stable scoring than just the like did it just pass the tests. So like we're moving in the right direction, but again like the model judging code quality I feel like can only go so far. Like the whole point of why models keep getting better and better at solving coding problems and why Fable is a little game changing and why like the sole like 5.6 family models are game changing is like they did get better at solving problems, but it's because of what we've done during RL. We've just like the RL runs and the problems have gotten more sophisticated and if the model could tell good code from bad code, I would expect to know that because it's writing good code in the first place. So like if you write something with Fable and then you have Codex review it, it's like, okay, if we have to do this review at all, like it does raise the floor, like throwing more tokens at the problem will probably catch all the small stuff, but I I don't know. I this is my like the the thing I can't quite like prove, but everybody I know has this vibe, which is like if the model knew what good code looked like, then it would be able to write it in the first place. We're all using the same freaking models. >> I mean like like you said earlier, it really depends on the state of the business, right? Cuz like I know people who can get completely one-shotted and just like spend 7 months building the perfect system, not get a first user. I also know people who like, you know, can do the business part, can do the marketing part, but then like you said, when it comes time to scalability, maintainability, they get [ __ ] because they, you know, don't put in the effort and they have no idea what's happening. So, I think it's like, okay, you have product-market fit, you have, you know, enterprise customers, you you want to make sure that you don't lose their trust, you don't get sued, all that stuff, right? If you don't have product-market fit, you need to move fast, you can rely on the models a bit more, still try to architecture it in a ways that like scales, still try to keep your grip on the code base, but like ultimately, if you have zero customers, first figure out if you're building a thing that's even worth building. >> Yep, 100%. >> By the way, I packaged the exact program design system that Dex uses into a single bundle. It's available as the first link below the video. It's completely free. Go grab it now. >> There's this thing that like Jake Nations, who is like at a time he was, you know, the number one cloud code engineer at Netflix, is my impression. I don't think he ever said that, but that that was what it seemed like. He was kind of like leading the charge on agentic coding way before it was like super mainstream, and he has this point of like as a software engineer, you have a lot of hard-earned intuition, whether it was through AI coding or non-AI coding, you know a bad pattern when you see it because you've been stuck trying to fix it at 2:00 in the morning, or you spent 3 days like redoing it so that it actually is like easy to maintain and stuff. And like if you are not leveraging that, you are leaving money, quality, velocity, the ability to ship high-quality like stability, you're leaving that on the table. And so, my whole thesis around all of this is like how can we take the like entropy-tangled like set of like what are models good at and what are humans good at and what can we trust models to do and what can we not and like separating that out and like what is the right workflow that allows you the human because we we agree there are things that the model can't do without a human. How do we organize that set of chaos into a set of steps or ideas or practices that let you bring your intuition to leverage without like slowing down like the idea of yellowing a 2000 line feature and then just reviewing all the code at the end. We all know that sounds terrible. Nobody wants to do that. So how can we how can we how can we like pull the intuition out earlier on in a way that is like easy, pleasant, fun, interesting, whatever it is for the human and then letting the model cook on the things that the model is good at. >> I think my biggest like contribution to this is trying to get more things into the code base. Cuz traditionally the code base is all about the code, right? But as you said that the models are getting better and there's a lot of different things that like normally would be like understood. Like this is our marketing strategy. This is our public image. You know, all of the engineers understand it. But you know, the models don't understand it. So like personally I'm trying to every single project even a small one just like /docs/adr and document all ADRs explicitly and like know this is a decision. We're launching the pricing that way. No, we're never going to change the API schema if there's a massive change we're going to ship /v2 to not break the contract. You know, and then another thing is /docs/external. For example, what is my virtual environment variables like? You know, obviously not the values but like what what's in there, you know? What does my payment processor setup look like? Which emails am I using for testing? Where do customers contact with me? Like all the external things to the code base so that if the models need to find them, they are in the code base as clear markdown files and they can use them as context. >> Yeah. And this is a really interesting like idea of context in general is like I I have found and like the way our product works is like it's all files on disk, right? When Claude's working with plans or technical design docs or whatever it is, it's just using the file system, like the thing that it's really, really good at. And then under the hood, we have hooks of like every time it edits a file, it syncs to the cloud or notifies the team, and people can put in comments, and the comments get pulled back in. Like all this stuff but like if you can make the environment like close as possible to what the model is good at, which is like read, write, edit, grep, bash, then you then then you've like made it very like efficient for the model to get that context. So pulling all that stuff into the file system is like super, super smart. I'm I'm sure you've been doing this for a long time. Like now it's kind of obvious, but like yeah, the that idea of like how do we put as much as possible in the code base so we don't have to be like, "Oh yeah, well you got to turn on this MCP server, and here's how you check the most recent issues." Like all those instructions about how to get the context are wasting like attention in your model while it's trying to do work versus like you can just do this deterministically. You can just literally every time you start a session, you have a hook that is like, "Okay, pull pull the last 10 century issues in case we need them." That's free. That's literally free. It's like code that runs. It costs, you know, you know, CPU cycles, but it doesn't cost inference. And you should save your inference for the things that the models like are really, really like needed for, which is like reasoning through these complex problems and working with you to like come up with a solution and then spit out all that code. >> So let's talk about more context engineering because we're kind of flirting with this topic, and you're one of the first people who coined it. So I guess so how does that change since 2025 to 2026? >> Yeah, so I mean I'll I'll I'll put the screen share back up. Um okay, so we wrote this paper in in April, March of 2025, uh about 15 months ago, called 12 Factor Agents. And this was the first time that we ended up talking about context engineering. And we basically had these like 12 different things about if you are building agents, these are things you should probably think about. I would say at least nine of these are kind of still relevant. Some of them you kind of get for free and they've just been like adopted into most of the harm harnesses, but like here's the the the most interesting one, the one that popped off the hardest was like own your context window and basically this idea of like everything that you do when you work with LLM's, whether you're writing a script that builds a prompt and sends it to the model and parses the output or you're using a coding agent and you're like, you know, your agent it's assembling its own context window, but you're telling it where to get stuff is like rag and history and memory and prompt engine. All of these things are just like tokens in, tokens out, right? The only way to get like better results from the model is to put more tokens in put the right tokens into the model, not more tokens, but like put the right tokens into the model. That's how you increase your chance and so like understanding that the only like primitive that you have to work with is this context window, whether it's like a list of messages or just one giant prompt, right? There's two ways to do this, right? You could have this like system user, assistant, tool or you could just literally have a system message and then the user message has some other formatted version, but your goal is like when you're building these context windows, especially for agents is like and like obviously you wouldn't do this today because like caching has become really important and this is going to bust your cash every time, but like with agents what's really what's really important is just like understand like what's going into your context window and you can do literally anything you want. Um and so you can literally just model this as state in a program. You can just turn everything that's happened into a prompt and dump it in and like if you understand like literally down to the token level that like, oh, if I put JSON into my context window, that's different from putting XML into my context window and it's more token efficient to do XML. Uh it's like however you can convey the information to the model in the most like efficient uh like basically like you want to make sure you don't have any wrong information. You want to make sure you're not missing information and then you want to make it as small as possible in that world, and this applies just as much to building like an AI pipeline in your back end to like how do you think about using a coding agent? And so this was April 2025, and then in like June Andrej Karpathy and Tobi Lütke decided that they were going to talk about this and I don't know. I still haven't get I tweeted Andrej every once in a while. Let me see if this tweet will open up. Yeah, here it is. I every time someone tweets about context engineering I I come and tell them like, "Oh yeah, this is from a while ago." but he never he never replied. So it's fine. Andrej, I forgive you. It's fine. You couldn't have known. Maybe it's a Newton-Leibniz moment. Actually Jeff Huber from from you know Chroma DB? >> Yes, I know it. >> Yeah, so he also posted the words context and like he posted a tweet in like 2 weeks after 12 factor agents came out. And I'm I'm pretty sure he didn't read it. It's like he came up with this he just posted two words context engineering. And that was that was the whole thing. So it was like I think it was like a lot of people were kind of coming to these ideas around the same time. But yeah, I don't know. That's like the the history of it. >> Yeah, I mean that I think that's same concept appeared through history like multiple times of science and technology where like multiple people came to the same realization at the same time because they're you know thinking about the same problems. But yeah, I mean I think like right now how you think about it because I think the the biggest you know it's like the curve of like the mid like the mid width curve you know with the people who are complete beginners they're like super short like fix this and they don't even provide a screenshot. They don't even provide any logs. The people in the middle like spend 30 minutes writing the perfect prompt and the people like who are like locked in they just like provide one screenshot is like oh it's in that file just read that file and fix it. Right? So like >> Yeah. >> how is the context engineering now? Is it really just like trying to have as high signal as possible with as little tokens as possible to not limit like what the model can do because like maybe the model comes up with a better solution. How do you think about it? >> Yeah, I mean it's like the the steering side of it is interesting. You talk about like you mentioned one thing that's interesting, which is like, okay, you have you can tell the model how you want it done, and that's like more likely to hit what you exactly what you wanted. And if you don't tell it how you want it done, there's a broader realm of probabilities, right? Of like, okay, it might do it in a terrible way, but it also might figure out like a better way to do it that you never would have come up with. And I I put that separate from context engineering cuz that's more just like how are you prompting and like what are you open to? But I think of the a lot of the concepts in in context engineering are still really relevant. This was an accident. I did not set out to like most things in AI are completely complete [ __ ] after like 6 months, but I think this one is tied like context engineering gets into like the physics of context windows and like how transformer attention works. And until we get like post post transformer architectures or linear attention, it's basically always going to be like the the smaller and tighter and more specific you can keep your context window, the better results you will get. Now, the the floor keeps going up, right? And so like I don't think super deeply about every token in my context window unless I'm doing something really really hard, right? It's like you said, if I'm doing a UI change, I'm literally just like, "Hey, here's the thing I want. Like, I trust Codex to go read a lot of stuff and figure it out." And if it reads a couple extra files that weren't relevant, like that's fine. It's like it's perfectly capable to go to 200, 300K tokens before and and not kind of like crash out on me. Um and this we talked about like the dumb zone a little bit with our context our concept and like I actually came up with this. I was sitting in the office with a bunch of engineers who were like a little bit new to AI coding. We were doing like a like a training session. I was sitting down and I was like I was kept pointing out. I was like, "Okay, cool. We're at This was with like Opus 4.1 and then Opus 4.5." I was like, "Okay, we're getting around 100,000 tokens, about 50% of the context window, 120,000 tokens. It's time to like take everything we're doing, like compact it into a whatever document we're working on and be like, "Hey, cool. Take everything we talked about, put it in a doc." And then basically like start a new session off of that doc and like keep working. And the the term I came up with was like, "Okay, cool. Like, the model's in the dumb zone now. That's why it's That's why we're not getting good results." And again, like, this is more like I think of this as training wheels. Like, if you have no LLM intuition and you haven't worked with agents for 70 hours, you can figure this out in about 3 weeks of talking to Claude for 70 hours a week. You can kind of figure out like, "Oh, yeah, no, this is off the rails and I should start over." Uh just by trying it a couple times. So, I would regularly go up over 200, 300k tokens depending on what I'm doing and like, "Okay, I have two It's like I'm too lazy to go like curate this into a handoff and then go start a new session." And like, compaction's getting a lot better as well these days. >> Yeah. >> Uh so, it's less of a like, "You must stay under 100,000 tokens or your" That's the peak of the midwit curve, right? Is like the the other side of it is like, "Just do what works." Like, you have the intuition, you know what's going to And if it's not working, one of the things you can try is start over and like reset the problem. >> Yeah. I mean, the two biggest faults is like as you said, you know, the end justifies the means. Like, just if you can get the result that you want it and okay, maybe use this model, use that model, use a different harness, you send a few extra prompts, it doesn't matter. There's many people who just spend like 80% of the of the time every single day building tooling and 20% like doing their [clears throat] actual work. Like, my goal is to not be that person at all. Like, I you know, if if I have the sub-optimal setup and someone has like one more token than me, I'm fine with that. I just want to build the thing. But, the second thing I would say is like the human It's the same in the human. Like, for me, like, I I can easily find myself working for 3 hours on some problem, then I go on a long walk and I'm like, "Man, this doesn't matter to the business at all." And like, that was my like dumb zone. >> Yeah. Yeah, humans have a dumb zone. If you're tired, your results actually It's really funny. Uh the other metaphor I really like is Calvin Frenchie, the guy who's like Codex, he founded Segment. He He His metaphor is like, the context window is like a It's like a student like when you're sitting and taking an exam, you kind of go through like the first half and you're like, "Okay, cool. I got plenty of time. Let me think. Let me work out these problems." And then you get like halfway through and you realize you have 5 minutes left and then you start like rushing through everything and like you'll see models do this especially like Opus 45 and Sonic 45. They had this thing of like context anxiety of like they would they would they would realize the context window was getting small and so they would start cutting corners. They would like scramble to do stuff and then they would just like give up. >> Yeah, I mean that's the biggest problem I would say. >> It's yeah. When when when when when you get tired and lazy then the results go down for humans and for agents. And I think that's my problem with all this like token maxing stuff. What you said exactly is like I don't need the most efficient setup. I need to care about the outcomes and the results. Do you ever read a book called The Goal? >> No. >> Okay, so this guy Eli Goldratt wrote this book in like the 70s called The Goal and it was kind of this idea of like It's It's this story of a a guy who's trying to like save a factory. And like I guess the way factories worked in America in the 60s and 70s is you had like all of these like stations, right? And this is going to get into software factories as well. You had all these machines and it's like, "Cool. A bunch of raw materials go in and out comes a car." Or out comes some part that gets shipped to another factory to make a car, right? And basically what you would do is you'd hire a bunch of MBAs and they would like get assigned a station and they would max the utilization of that station. You want to make sure like we're going to turn this thing on. It costs money to spin it up and every minute it's sitting idle, we're wasting money. And so everybody would like optimize their stations and what that would lead to is like well different stations had different like times it took to process stuff and there would always be some bottleneck and in front of the bottleneck you would have a bunch of work in progress like stacking up which if you're running a factory is very expensive to store work in progress. And so it's basically this story of like there are inefficiencies that are not bottlenecks and the idea of like token maxing because of the subscriptions that are just like I need to spend all I need to exhaust my cloud stuff every 5 hours. >> end up in the hospital. >> Yeah. Oh, that That That was crazy. I don't know if I fully buy that. I don't know if that was a meme or whatever, but I hope everybody's okay. But yeah, that's the kind of the software factory token bank is like how much can you spend? How much like stuff can you actually extract out of the machine and turn it into code and like there's something to be said for like, you know, try to push the frontier, do the biggest, most ambitious thing possible and see how it goes. But then you should probably walk it back from there because it's like if the bottleneck is code review, like adding more coding agents isn't going to solve it, right? The bottleneck like you need to figure out how to solve the actual bottleneck versus like, "Okay, my agent station is really efficient and my deployment station is really efficient and every time I make those faster, you're not delivering value faster because the thing that matters end to end is like value delivered to users. That's what gets people to pay for [ __ ] >> Yeah. I mean, it's it's like the, you know, the best example who does it right is like it's Elon, but he always works on the biggest bottleneck and then like he doesn't mind if somebody else in the company does something else even though they're probably not going to do it on his level. If it's not the bottleneck that, you know, doesn't matter. Like if they do it on 80% it's fine. He needs to work on the bottleneck. And the same thing applies to like, you know, building a software company or a software factory. It's just like if you're improving your multi-agent setup with like fancy routing and delegation, cool, but like, you know, the production is down or something. Like, you know, >> We have a We have a joke in our office that is stop playing with your coding agents and get back to work. It's like it's so easy and it's so fun to build the thing that builds the I mean, this was me even before AI. I was My first job out of college I was working at a like a social media analytics company. I had a lot of data. I had a lot of good engineers, but I noticed that like the most important person in the company was the guy who built the dev platform. The way that you ship, the way that you deployed your like sandbox environment to send a preview to a product manager. The one who like would like knew how to debug the database Not the database itself, but the rollout procedure for it. And I like since then I've just been obsessed with like every time I've been a company and the CICD is slow, I'm just like, "Oh my god, we need to make the CICD faster. We need to make the cycles faster. Oh, we're releasing once every 2 weeks, we need to release every day." Like I've been obsessed with I've been obsessed with software fact I I didn't know that's what they were called, but I've been obsessed with this idea of software factory since you know, the for the last 12 years or whatever, however long I've been working, 15 years. And uh I would often have be on the other side of this conversation of like, "Dex, I need you to stop trying to make CI faster and just go ship the tickets that I gave you. Uh please." Like I would have this conversation with my boss like multiple times at multiple different companies. And I was like, "No, we have to build the thing that builds the thing because that's going to speed everybody up." And it's like I think the hardest lesson is like there are inefficiencies that are not bottlenecks. And you may see things that like make your skin crawl. You say, "That's terrible. Why are we doing that? We could do be doing it so much better." And like if you care about entrepreneurship and building things and all of this is like the most important skill you have is like zoom out, ask yourself, "Is this a bottleneck? Can I just hold my nose and like focus on the thing that really matters?" >> Yeah, as you get to like more successful, whether it's in business or in, you know, software, you will have fires burning. And you need to be comfortable like letting the fires burn. It's like which of these is actually the wildfire that's going to burn down the whole city and which are just like some small embers that you can leave to the side and leave for later. >> It's like, "Okay, cool. Yeah, do we go do we go shut off the gas leak in city hall or do we go put out this fire over here?" It's like, "Well, that one's you know, it's not burning yet, but if it does, it's it's it's it's like it's almost like also like expected pain, right?" It's not It's not just like, "Okay, I don't actually know." And this is the hard part is like holding all these probabilities in your head of like what's worth working on. Is uh is is uh is it crazy? Did you ever play uh did you ever play StarCraft? Were you ever like an RTS guy? >> No. I'm a too young for this. >> [laughter] >> The first the second one came out in in like 2012. I don't Anyways. >> I mean I play I play World of Warcraft, but I feel like StarCraft is like one generation before that. >> Yeah, it was and it was a different Well, so like StarCraft II I think came out in 2012, but it's like >> Okay. >> It's a game that teaches you to make hard decisions with incomplete information and you are constantly making hundreds of decisions a minute and like you don't know everything. You don't know You only know what you've seen and you say, "Okay, I saw him do that and I saw this over here, so that means there's like a 10% chance he's doing this and a 30% chance he's doing this and a 70% chance he's doing this." And so like my best thing is like, "Okay, if I do this it has a 40% chance of beating this and a 20% chance" It's just like this this game of like stacking probabilities on either side Anyways. It's uh it's it's it's a good game. I don't recommend It's incredibly stressful. But uh yeah, this that skill of like knowing what you've seen and making bets and and and making I mean I don't know. Founders talk about building is like investments and tech debt. I think I was talking to somebody at dinner last night. He's like, "We are not making investments in our infrastructure. We are not making like we are we are gambling. We are all gambling all day every day trying to figure out what's my highest like expected value." >> Yeah, or like you know, should I even be building this company? Will I spend the 6 months on it? Will that be completely terrible but then 6 months will I just be replaced by OpenAI or Anthropic? But all of these are like bets, you know? And you cannot make too many wrong ones or you'll be irrelevant. >> Well, are you having fun? >> I am. >> Yeah? That's a that's a good start. I mean if you're having fun and you're learning then it's like, "Okay, cool. Like so so what?" I mean crazy things never got like 20 years ago there was a hundred startups that exist today that every I guarantee they had, you know, nine out of ten of their invest investor pitch meetings people like, "Nah, Google's just going to do this." And guess what? Google did. >> Yeah, yeah. Yeah, I mean you can see it even even this generation I talked with Magnus from Browser use and everybody was like, you know, Anthropic's already building this, there's no point. And you know, they managed to do it better than Anthropic. And that's literally like year, 2 years. So, yeah, even in today's today's time, like most most of likelihood is these big companies are not focused on your small niche idea and you you probably still should build it if you really want to build it. >> One of my favorite like sound bites on this is like, you're not actually competing against Google or you're not actually competing against Anthropic, you're competing against a product manager at Anthropic. And like, okay, yes, very very very very talented people are there, like you're not it's but like especially the Google version of this is interesting. It's like, okay, there's a there's a team inside a large org that has a lot of rules and red tape and like things kind of move slowly and it's like, I don't care how much money they have or how much distribution you have. A small founder and and a small team who really gives a [ __ ] can out-execute most teams inside a giant enterprise and like, what OpenAI and Anthropic have done and I've talked to a lot of people who who are former employees who work there is just like, oh, they've done a very good job of trying to keep that startup energy and help people move as fast as possible and hire really really like Boris and Katya are [ __ ] insane. They're so they're so good at what they do. So, it's like, okay, that's a PM that is like a little bit a little bit you know, I would think twice before getting in the weeds there, but it's it's very easy to just be like, oh, it's not worth building anything, the labs are just going to win and like, I don't know. That's not a I like to build things in the vein of like, they're they're aligned with the world I want to live in. And the world I want to live in is a world where like you have large companies and small companies and innovation can exist and like we can all solve problems together and like, I don't know. I'm friends with all the coding agent founders in San Francisco. We're all competitors, but like, it's a huge market and in my mind it's like, it's not founder versus founder, it's like founders versus the incumbents maybe or like founders versus people who hate innovation or something. I don't >> Yeah, yeah, exactly. The people who people who hate technology and are like uh stuck in the past. It's like I think it's like in the AI space the more I you know, the more people I talk to I realize it really is are you a true true believer or not? And so many people are here because of like this is the next, you know, wave. This is where money is. This is where, you know, hype is. But like as you talk to people a lot of them are not really true believers. They're like not don't truly believe in technology, right? And like even when like a, you know, new model comes out or there's a new tool instead of like testing it, embracing it, giving it a shot they're like instantly like ah it's another one of these, you know, or or it's too expensive this model. It's like yo like guys we're inventing the future. >> and it I tried it a year ago and it sucked like oh well. >> Now, if you want to actually implement everything that me and Dexter talked about the entire program design system that he invented is available for free in the first link below the video. So, go ahead and click on it. Just enter your email and you're going to get it received on your email. Again, completely for free. So, go get it now. >> Um yeah, I mean it's it's a maybe a good note to like zoom out on and maybe maybe maybe like start to close out on but like this idea of like, I don't know, I was in an event last night. Um it was like Dwarkesh interviewing Dr. Fei-Fei Li and then Michael Grinich from Work OS came out and he was interviewing Dwarkesh for a bit and like I don't know. We're all here in San Francisco and like it's very easy to get sucked into like what's the what's the coolest thing that we can build? What's the hardest problem we can solve? How how how dope can we make it? How fast can we go? And it's like I don't know. Grinich did a good job of kind of pulling us up a level of like, well actually like AI is the one of the most unpopular things in the world right now. Uh slightly less unpopular than AI was like ice and slightly more unpopular than AI was like the Iran war, right? Uh but it's down there and like most people are distrustful of it and like I think it's it's important as builders to kind of like zoom out now and then and just realize like, hey look we need to think about how this technology works in the I mean Dwarkesh is talking about like, hey in 5 and 10 years there's no labor, like there's nothing to do, like how does the economy work? Like do we need to go talk to PhD economists and think have have these people thought about like how does the world function in in that in that new paradigm? And like these are things that like nobody has answers to and until you give someone a compelling someone who's not technical and not super AI pulled until you give them a compelling answer is like this is how the world will work and it will be good and everybody will win other than the very basic like hey industrial revolution life got a lot better. I mean it got worse for some people in in lots of ways but like overall we have more abundance now than we had in say the 1500s and I I don't think that's satisfying enough an explanation for most people. >> Yeah, people care about am I losing my job or not? >> Yeah, am I going to be able to feed my family whatever it is and uh so I I I think it's still worth it. >> easy to get stuck in it. If people especially like you know on Twitter or San Francisco which I still need to visit San Francisco. I've never been. Uh I think I'm going to Yeah, I think I'm going to visit in like late September maybe early October but uh it's easy to get stuck in the bubble, right? Whether it's on Twitter or in SF but like most people they've no clue. The only AI they use is like the Google AI search and they don't want to lose their job, right? So if you talk about like oh yeah, we're going after labor like they they will instantly hate you. >> I mean I don't know why they keep saying that. It's like I guess it's better to say it and be wrong than to not say it and be right but uh yeah, we'll see. I don't know. I I I kind of have this theory about like how to think about AI and like there's there's lots of like crazy things that you could panic about, right? There's like okay, if we hit this like singularity and there's complete abundance and no one has to work anymore and everything's just great. Like that's one world the like incredibly good version of the world. There's also like a percent chance that like we screw up alignment and we all get like we're all living in the plot of the Terminator and the machines are just trying to like wipe us all out and then there's like this there's like of like how bad the outcome is. And there's like this middle line going through like the center of it of like things just keep getting better and they might get better 2x faster or 1.5x faster every every year every month or whatever it is. But it just kind of stays stable and like nothing too crazy happens. We don't get completely destabilized. And whether you think the good scenario is 30% or 3%, whether you think the bad scenario is 30% or 3%, it's like the only world that is actually makes sense to plan for I think is like the middle path. Because if either of those other two things happen, like nothing you do today really matters unless you're at a lab working on alignment. >> Yeah. I think that's a that's a very valid point. And I think we can wrap it up on this point. I want to be respectful of your time. We we already crossed an hour. >> fun. >> Yes. Where should people go? Where should we send them? >> Uh follow me on Twitter Dex Horthy d e x h o r t h y and check out humanlayer.com. Um if you want to learn how to do this. >> I'm going to link both below the video. Again, thank you for your time and have a good day. >> Good stuff, dude.