How Stanford Teaches AI Coding Now

0:06 I made it into the Whimo with Mika, Stanford lecturer, head of AI at Monaco. Here we go. >> What have you been doing with this Stanford course of yours that is very

0:20 famous now? >> Yeah, it's been a it's been a fun time. So, we're about this is like I think week five now of the Stanford course. >> Yeah. and um

0:29 >> titled the modern soft >> the modern software developer. That's where it's at. Yeah. >> And you have an email that goes out in >> I have an email. I do. I do. Yeah. So,

0:36 there's a newsletter uh the modern software.dev. I have the domain, all that good stuff. Uh but the course has been a ton of fun. I mean, it was um this is the second year that I'm doing

0:46 it and compared to last year, there's just like a huge overhaul of the curriculum. Um, pretty much none of I mean I I literally I think 80% of the stuff I threw out over over the last

0:56 like two months >> cuz just once Opus 45 came out December, you know, December/ November to December of last year. >> When was the last year's course?

1:06 >> Last year's course fall. So it's always in the fall. And so the the last course uh in 2025 was September through like early December of of 2025. And so for that like window of time, nothing

1:18 changed too much in terms of the model. Good. I was good. I was good. It was stable. It was like relatively stable. >> And then December rolls around and then uh you know things blow up in terms of

1:30 capabilities. People jump on the cloud code bandwagon uh through like that's basically what everyone did over the over like December break, you know, during the Christmas holidays and New

1:40 Year's. People came back early January and they were just there was a manic craze around cloud code. Um but it was it was I think it was pretty justified. I mean the the models got so good that

1:49 we could start doing some really interesting like long horizon agent tasks. >> Um anyway, so because of that I had to like rethink the entire curriculum. It's

1:57 just like well what's now what's now happened in the last like 9 months since Cloud Code uh became >> So now you're talking about harnesses you're talking about

2:04 >> harnesses loops software factories. We're talking about like you know fully automated AI code review when not just hey you know add a GitHub app and and like have it do your code review. It's

2:16 it's more like what how do you do like auto auto approvals? How do you actually do like merge conflict >> merge conflict like all DevOps stuff, agent readiness in in in code bases. So

2:27 it's a ton of stuff. Um >> yeah, it was is it was a pretty big overhaul. It was most of my summer um this this time around I think uh oh and then the other thing I'll mention that

2:36 was also fun is I'm having all the students do like a um open source contributions as their homework assignments. Nice. So they are there's no last year I was just coming up with

2:45 these random exercises. I'm like oh like here's a codebase like add add your add a code review use a prompt. Um but this year I was like all right here are these like open source partners that I was

2:55 able to get on board with the project like a lot of the top open source projects out there and like can you can like my students contribute to your repos as part of their homework and

3:05 >> like sure we love slop. >> Yeah. [laughter] Not quite not quite. Yeah. I mean, okay. So, so this is I'm I'm like I feel like I'm a pretty generally a pretty chill

3:15 instructor. Um, but there I was like if if I get a maintainer messaging me saying like you guys need to stop this slop slop grains. I'm like zero. That's a straight zero on that assignment.

3:25 >> Um, that is where I'm I need to draw the line somewhere. Fail >> fail. >> Otherwise, I'm chill. But like I, you know, I had to I had to go there and and

3:33 and I, you know, I made these arguments for the students. Like these are these are like, you know, Stanford students. They're they're academically trained. they like good CS fundamentals. Um, so

3:42 you you know like it's not going to be slop and I'm going to teach them how to not do slop. So if that anyway so all to say like that they have to do real contributions.

3:51 >> How are you teaching them to not just merge slop or at least submit slop? >> Yeah. No. Well well part of it is you know through through force through will through will no

4:03 >> read [laughter] every line of code that your agent produces. >> Exactly. No. Uh it's it's like I I I'm teaching a lot of like like advanced context engineering stuff. Uh teaching

4:12 them how to like onboard into code bases really quickly. So if you have these massive repos you've not contributed to at all. How do you get up to speed on like the most important parts of the

4:20 architecture? And then when you get these issues that the the maintainers have put up and designated for the students like how do you scope the the work? How do you then go and actually

4:30 start um you know gathering like context for your agents that they need to use? How do you connect the right MCP tools? use the right skills. Um, all the stuff that has become like incredibly not easy

4:40 but like way way easier to do. >> Yeah. >> What are some of these examples? Obviously, it's not it's a what 8week course or 12 week 10e course.

4:48 >> Condense some of these best practices into how you get up to speed because how I would do it is just say what is important >> and that's it.

4:57 >> Yeah. Yeah. And I think like a lot of people start there. Um I think that the what I'm trying to argue is that there's still value in doing some version of I think you know in my class we call it

5:08 like repetit. I think uh DEX and others would call like RPI like some version of like research you know research plan implement test kind of like that loop. >> Um and I think just yeah so so at its

5:22 core some version of the RPI loop um or using superpowers like if you you know the the what is it uh The other victim of these Whimo things was Jesse. >> Jessie. Jesse. Okay. Super. Nice. Nice.

5:35 Yeah. Jesse's I mean that's that's a fantastic project. Um incredibly simple but but so I mean it's become one of the most popular projects of all time. >> The fact that it still is popular.

5:45 >> Yeah. Yeah. >> After a year of all these advances, it shows you how valuable it is. >> Totally. Totally. Yeah. He's I mean I think he does a great job and he's such

5:52 a humble guy too. Um but yeah so like some version of that framework of you know gather the context for your agents like research at the beginning don't just like jump into the implementation

6:02 review a spec you know actually have the spec be be reasonably detailed and um make sure that you have a way of verifying your outputs so that if the agent runs you know for a while that it

6:13 can check against >> you know using the browser or checking like against end to end tests or like smoke tests or something. So, >> can we go down that rabbit hole? Because

6:20 that's for me one of the biggest things that is >> where I feel like I can spend the best use of my time. Yeah. >> Is setting up verifiers. Jesse told me

6:30 he created something like prove it in a movie as a skill and it makes the agent go through and create a movie for you or a like a loom. >> That's funny. and it records itself

6:43 creating the or walking through the app and showing that it's functioning correctly. >> Yeah. >> Have you seen other like ways of folks

6:52 creating verifiers? >> Well, well, there's what I've seen a lot of is people getting creative in how they how they uh represent information that that is that the diff uh the diff

7:02 of the code, right? So like I think Andre Karpathi had a whole post around how he there's like this format from like the 19 like 50s or something or like how people made these like

7:11 architecture. Have you seen this? >> NASA one random number. >> Yes. Yes. Yes. But but you'll see it and it's like a very nice like visual

7:20 schematic of of kind of the core important parts of what happened in the codebase. So I think stuff like that I think is super super interesting. I think yeah the screenshots recording

7:30 flow I think is going to become more and more popular. um you know doing some version of like using playright with MCP. >> Oh and you know what I saw recently that

7:38 is a fun one is that web MCP >> has now merged just debugging tools. So like for you can stand up a web MCP server. >> Yeah.

7:50 >> And it's only for debugging. So your agent can now use WebMCP on a web app. >> Yeah. in like computer use mode >> and then it can use it for debugging specifically.

8:03 >> Yeah, I mean that that stuff is super powerful and and I I I suspect I mean there's entire companies around this, right, that are just trying to >> improve the verification loop and close

8:13 the loop entirely. And you know, I think all of them are going to become all of them are going to start knocking on the door of software factories basically. Um I I think we've seen this we've seen

8:23 this game play out enough times with with ML and MLOps and LM ops is like everyone starts in one thing and then starts expanding >> expanding because they're like wait

8:31 actually we need to go upstream. The real problem is upstream. >> Totally. Totally. Yeah. And downstream now. Yeah. So it's like you you start as a feature store and then you become data

8:38 bricks platform. Yeah. >> Yeah. Or something like that. >> Yeah. It always I think that's you know that's it. I think it's it's the typical like the company bundling on bundling of

8:50 like you know that always happens. Uh are you going to be a Craigslist or you going to become like an Airbnb, right? Where you you and I think it people will swing back and forth on whether or not

8:59 they think one is better than the other. But I think we're going to see a lot of that here, right? It's like start the air vacation but also we'll own the coding agent and then you have these

9:05 companies like Factory and Warp that are like >> like we're going to own everything. We should own everything, right? >> Um so it's all like really interesting

9:12 stuff. It's all exciting. It's been it's been a ton of fun. Uh Boris came last week. >> I was going to say you've had some really cool guest lecturers.

9:21 >> Boris, the creator of Claude, one of them. He's got a lot of hot takes. He's got a lot of I think one thing is nice that he just talks about how he uses it and it's pretty advanced because

9:32 >> he has no token costs, right? So his whole thing is like how can I saturate this to the max? >> Totally. Totally. And so then we get like a very distilled version or we can

9:43 take some of those ideas and say, "Oh, well, I want to try and implement that." >> Absolutely. >> Like one thing that I I saw he was saying recently was how he he never if

9:53 he does a one-off prompt, he's doing it wrong. It should always be like talking to a loop. >> Yeah. >> And then letting the agent prompt

10:01 itself. >> That's right. That's right. Yeah. Yeah. Like the meta prompting. I was uh I was sitting with him uh last week and my first question to him was like how many

10:11 agents do you have running right now and he was like I don't know like he said this so he's like I don't know like thousands or something. I'm like >> as we're sitting here, you know, like

10:19 chatting it up. Uh very inspiring like to talk to him because you say this is what >> this is what it looks like at the very frontier like the very very frontier

10:28 where where yes, no token cost, so obviously they they can just do what they want. But but on top of that, I mean just being very experimental with their philosophy, you know, where

10:37 they're like, "Hey, we're going to try and solve math problems. We're going to we're going to try and solve, you know, like really difficult complex things by just throwing almost like brute forcing

10:45 but intelligently brute forcing things, you know. >> Um so yeah, Boris is is it was a fun talk. I'm talking with Eno this week from factory

10:53 >> factory. Yeah. Factory. Um Lee Robinson was two weeks ago. He was uh I guess now he's at SpaceX, but he used to be VP of Devrell at Curser. >> Curser. Yeah.

11:03 >> Super smart guy and super nice. Just like the nicest guy. >> And what are some things that you took away from that little lecture? Yeah. The the Boris one or the Gro the the lead

11:12 one >> one. >> Yeah. >> I've seen him on X occasionally, but I haven't

11:18 >> It's not as much as Boris. Like Boris puts out some and it's like Karparthy type stuff. >> Yeah. Yeah. >> Wow.

11:23 >> Yeah. Boris got got super popular after Opus 45 and after like >> after your course came on your course last year and then >> that wasn't that wasn't the thing.

11:34 >> What a [laughter] surprise, dude. You became popular. >> Yeah. That was uh that was you know what I I there's there's two people that I'm that I feel like I discovered before

11:42 they became big. It was Boris and and Lady Gaga. [laughter] >> Oh yeah. Lady Gaga you made her career also. Why did she come on your podcast?

11:52 >> Okay. I I didn't lay anyone's never saying I'm not responsible for anyone's career. I'm just saying like Lady Gaga in 2009. I was like you know this this poker face stuff. I'm like there's

12:01 something here. There's like people aren't seeing this yet but I I think there's something here. [laughter] >> Poker face. was already popular, man. What do you mean that was

12:10 >> I I was I like I'm telling you, man. It was you you had to be there. It was like, you know, just Just Dance. Remember Just Dance? Like that her first OG. That was the single. Anyway,

12:20 >> you were on board with it from day one. >> That's all I got. Those are my two claims to fame. [laughter] >> Well, I'm glad that Boris's crew, you could play a part in ming.

12:29 >> No, no, >> put him in your newsletter. >> We all We all owe him a a big debt. He's He's an awesome guy. The coolest guy. No, the coolest guy.

12:37 >> Us peasants [laughter] >> talk about some of this stuff. >> But no, what was the one with Lee? What did you see? What did you learn? >> Lee was awesome. I mean, Lee was just so

12:45 so cool. Um, so Lee was talking all about uh like basic Grogbot. >> Yeah, >> I don't know if you've used like the Grogbots and these like Instincts and

12:54 >> I've used Instinct. I've used Dots. Um I haven't used Muse, but I will say I do enjoy it and it's becoming more and more all I use. Yeah, I I think those those products are awesome.

13:07 >> Yeah. >> Awesome. Just >> I've heard Instinct is much more proactive. And what I've noticed with DOTS is it's kind of proactive, but not

13:16 really. It's just like, oh, I found this thing in your email that maybe you want to check out. It's like, okay, that's cool. But >> yeah,

13:22 >> it's not like the instinct one where >> they really go hard on like, hey, you went to a movie. How was it? >> Yeah. >> Okay, I guess you're my best friend now.

13:32 Am I in love? Like [laughter] >> you're listening to me more than than my >> totalent, but I just saw the funniest meme where somebody said, "My wife and I share the same OpenAI account." And I

13:47 just put this in as the like system prompt and it said, "Anytime Jane asks about marriage advice, Rob is always right no matter what happens." >> That's funny. That's so good. That's so

14:00 good. So that's what we've come to. >> That's I mean yeah these this like this subversive deception in in marital [clears throat] context. Um yeah, no it

14:10 was um no but so he was talking about basically his entire talk was about um the harness and and and how he how they make grobbot essentially >> and theirs is a little different where

14:19 you have to create separate like roles and bots right and so it's new threads each time >> I I one thing I haven't entirely figured out with the system is like how

14:28 important that is of like creating of the separate personalities because you can also just now they do have the concept like the main bot system >> and so it can throw it off too

14:37 >> it it should be able to and so I' I've made separate personalities, but I also don't know if it's like strictly necessary. >> Um, anyway, I think they're awesome

14:44 systems. I think this is such a step forward than where we were before. Um, >> not to interrupt, but there's open office. Speaking of which, >> now like we set didn't set that as the

14:55 destination, but uh, [laughter] >> sorry, sorry, sorry. I'm not supposed to. >> Oh, man. >> We We can run that again if you want

15:05 from the top. >> I'll edit that out. Don't worry. >> There's [laughter] There's OpenAI's office. >> Oh, nice.

15:10 >> Yeah, nice. >> So, anyway, Grockbot. Speaking of which, the antic the >> Yeah, the the arch nemesis. >> Yeah, exactly. Nemesis. That's the word

15:19 I wanted. >> Well, Dots, formerly known as Open Claw, I assume, right? It's got to be >> in a way. Yes, >> it must be. I I I couldn't imagine

15:26 either way, but um anyways, so he he he basically it was like a it was a 45minute primer on like modern Asian harnesses. It was incredibly well deta like very detailed. Uh he talked about

15:37 everything from computer use and how you implement that to context engineering to uh the tool calling how they how they you know give the agents the illusion of infinite context

15:47 >> by by doing just aggressive sub agents basically. >> Um anyway it was incredibly well done and and Lee I think his you know his one of his superpowers is just being such a

15:56 good educator. So he was just distilled very well. Um, >> fun, engaging. >> Fun, engaging. Just like super dense, but in like a good way in like, oh, this

16:05 is this is this this feels like you're you're giving me good signal like you know, signal to noise ratio was was through the >> Did you talk about how they do

16:14 compaction? >> Yeah, he he touched on that. He definitely touched on that. >> How are they doing it? Cuz that one is tricky.

16:19 >> Yeah, I mean there's there's different techniques, right? But, um, at its core, they have file systems. they you know regularly compact when they detect things are happening and then they like

16:29 write things to file systems and read them at at the right points in time in the conversation. So you know I hate to like go into like the exact detail like what is their what is their policy when

16:38 they compact but um I think at its core it's all some version of that really like um any of these agents I think follow a similar sort of set of principles. Have you been staying up to

16:48 date with any of the companies like that are trying to do file systems in a new way? >> Uh not like is are these the ones beyond the ones that are doing like um like

16:59 agent sandboxes? >> Yeah. >> Sandboxes are also another one that is very fascinating that that I would put in its own category.

17:09 >> And there's like file systems and then there's almost like we want to replace git and we want to do things differently in that regard. >> Yeah. Yeah. Yeah. Yeah. So, so yes, the

17:18 first um yes, the last the middle. I didn't know there was like separate is is the I don't know what the middle categories or if that's >> they probably do more. It's just when I

17:26 talked to them like the I talked to this one messa.dev >> and they were trying to rethink file systems because they're like file systems are all you need.

17:37 >> Yeah. >> Especially with these agents. >> Well, tell me about the git. You >> Yeah. Yeah. >> Yes. To the first, no to them.

17:44 >> Yeah. Yeah. Yeah. I mean, well, there's um there's sort of like I guess the the overarching reason for these companies existing in the category of like new gits is

17:53 and we saw this even last week. I think anthropic literally put out like a blog post around their home. >> They rebuilt it. That's so wild. >> The version control system. Yeah. And it

18:01 it all kind of makes sense. Well, one, you know, we've all been on GitHub for the last what 105 years and it's just it's kind of barfing a little bit. Um >> just reliability.

18:10 >> Reliability. >> Floatware. >> Yeah. I mean, and look, it's it's understandable when you have these literally astronomic curves of of growth

18:19 and commits and and um so it's >> it all makes sense that that would that they would receive that strain, but I think there's a broader like when I talk to like startups will pitch this to me

18:30 um as you know uh GitHub runners are just like not optimized for like agentic workflows. said you should you should like run on our runners like you like basically our machines for for that are

18:39 more streamlined to like these agentic processes and it all makes sense right it's like GitHub is like the GitHub and GitLab and these these kind of companies they they were the best solution for

18:49 everyone right >> but but not every workflow and workload is exactly the same right um so if you if you build these mega behemoth companies toward the median you're going

18:59 to miss stuff on the fringes and right now we're we're we're at the point where the fringes are maybe becoming more normal but still like the sphere of volume is definitely like a fringe

19:07 problem. Um >> yeah, >> you know, at least like hey 10xing number of commits, 10xing number of PRs, which most companies are starting to go

19:14 through that right now. So it's all fascinating stuff. Um but yeah, so so there's there's companies that are just like hey we're going to make your runners faster and then there's ones

19:21 like we're going to rethink the entirety of the the CI experience and the entire build system and all that stuff. Um, TBD, TBD, I mean, >> how much of your time because I imagine

19:34 startups are hitting you up all the time to try and be like, >> "We want you to use our product. You are the head of AI at a cool company." >> Totally.

19:43 >> Yeah. >> What how how much do you entertain that because you're actually interested and maybe it could have value versus you actually then later you have to work

19:52 once in a while like that. >> Yes. Yeah. I can't just take Whimo rides all the time. Uh, [laughter] >> no. Okay. So, so it's it's a really good question actually because I I feel like

20:04 I'm a pretty curious person. So, I love just kind of being on the having these conversations around technology and and seeing what people are coming up with. And so, I I definitely probably have at

20:13 least like one to maybe two of these a week. >> Um, but there I think where there's like a point of diminishing returns for me is when I'm hearing the same pitch over and

20:21 over, right, where I'm like, "All right, this is I see your >> not so novel." >> Yeah, it's not novel. like I I see your paragraph. I see your landing page and

20:29 like I I feel like I've I've heard this. So there's like a certain kind of company that I've heard probably enough times that >> which one

20:36 >> um and not not to ding them that this is not going to work but but in like a I do hear like this flavor of a pitch which is like hey you um we we collect like feedback um like you maybe you insert a

20:47 little bit of JavaScript on your app. We collect feedback. We trace all your LMS uh all your you know all the the requests and responses and then we optimize uh like we basically like on

20:58 the fly train models based on your request responses. So because we're tracing everything that's happening on your app, we get all your data essentially and then we know hey you

21:07 shouldn't have to go for this kind of request to GPT soul. You can go to like a very lightweight >> I don't know like a type safe model or >> you've seen a lot of those.

21:15 >> Yeah, I think quite a few of them actually. Um, and and it all kind of makes sense there. They're just I think that there's a whole new I mean a whole new category, but but it's becoming more

21:24 popular. There's this like open weight. If we expect that these these frontier models are like pretty expensive and most companies are getting hit with the cost and like, hey, we can't this isn't

21:34 sustainable. Um, we want to keep tracking and keep 10xing our usage, but we can't have our token cost 10x as well. So they look to these alternatives both from a coding perspective but also

21:46 from like a product development perspective is hey can I take a quen model? Can I take a Kimmy model? Can I take a GLM model? Like fine-tune it on something smaller or smaller weight be

21:55 faster. It'll be more it'll be cheaper. Um and I don't maybe compromise on performance, right? That's like the the the ultimate pitch. So this makes sense. This is a broader trend and I suspect

22:03 we're going to see a lot more of this. Um it's just I uh it's I think it's a bit hard to differentiate. Also, I think that the financial piece is a very difficult one. Yeah. It's maybe not for

22:18 an engineer the top of the pile of priorities, >> right? >> Because there's also things you can do when it comes to just optimizing your

22:27 skills and how they take in the context that can save you money, too. Or just a simple router. >> Totally. So you don't have to be optimized to the gills on your uh

22:43 cost-saving Yeah. >> analysis or ways of doing it. >> So yeah, that one's I can see that being a hard sell. >> Mhm. Mhm. Today traces I mean you maybe

22:54 heard this or if you haven't heard this, this is going to be a hot take. Traces are like the new oil, right? Like >> like company traces or data traces are are where like a lot of the juice is.

23:03 It's it's like it's like the modern incarnation of like training data. It's like you know companies training data is the new oil right um and so there's so much value in traces because that

23:12 effectively captures all of your customer interactions all of your like app level interactions and so when a company comes to me and says like hey put this little bit of JavaScript

23:25 and through this this additional layer your your your request supposed to go like you're just capturing all of our traces as well and I just I've kind of got to a point where uh we have a

23:34 tracing solution that like for evaluation and observability that of course captures our traces because that's what we host and you know brain trust in this case um and we also use

23:42 judgment labs for this but I'm not I I don't feel like I'm at a point where I want to keep just like willy-nilly having everybody >> trying

23:50 >> yeah have this yeah try a new one like oh now now it's routing to these guys you know and these guys and these guys >> incrementally better >> right increment is so I'm just it's it's

23:59 a tough cell but but this is I feel like that's the equivalent of when people started doing like feature stores in like 2020 so any of them and you're like is this really needed?

24:08 >> Totally. Totally. And and >> what about MCP kind of analytics and understanding how people are using your MCP server? >> Yeah, I think

24:15 >> feels a little different. >> It's it's it's somewhat different. I think that one ends up being more um I mean that's an interesting category. I think that one ends up being useful for

24:25 coding agent optimization. Um maybe less so on like product optimization. I've yet to see as much of that being applied to like product MCPU categories. Uh but definitely for I guess it's this

24:36 emerging trend of like agent X like agent experience you know you may have heard of this. It's like how do you optimize your MCP tools and your MCP uh servers so that an agent it's easy for

24:46 them to identify what they need to do and they can do it as quickly as possible in as few hops as possible essentially. >> Well and the the pitch that I've heard

24:53 is that you generally don't understand why the agent is coming to your MCP server because you don't know what the prompt is, >> right? you just know that it's trying to

25:03 call some tools, >> right? Right. >> And so, >> right, >> that doesn't necessarily need to be

25:08 fixed with bringing on a new SAS provider, right, >> that captures that information? You can build your MCP server so that >> you ask the agent, hey, yeah, cool.

25:18 You've got this tool call happening, but you mind if I get a little more information here? >> Right. Right. That's right. That's right. Yeah. I think there's an evolving

25:27 category, right? I mean, we we'll see where it all kind of shakes out. Uh I know that some of these providers are in the business of you know effectively they just run these like these tests on

25:38 your agents with your servers and say like how how easy how cost effective is an agent when it's making mcb tool calls how many tool calls does it need to make to to get to a certain answer um how

25:49 many errors is it hitting when it's making those tool calls you things of that nature. So, I think that's it's a cool it's a cool concept. It's a cool uh it's a cool category to I think be in.

25:58 We'll see what what shakes out as like a standalone product. >> Do you guys you have an MCP server, right? >> Yeah. Yeah. Yeah. For sure.

26:05 >> And so talking about Monaco now. >> Yeah. >> What are some things that you're doing in the AI field or just like the agentic field?

26:14 >> You have the MCP server. You also have agents inside of >> Yeah. >> Monaco. That's right. That's right. Yeah. So, so we've we've built our own

26:22 like agent agent builder and we are our our pitch is that our agents are the bestin-class for go to market use cases, right? So, we're we're a sales company. We're we're in the business of helping

26:34 um we have a world-class go to market team members and we're in the business of trying to distill the subject matter expertise of these world-class gotom market folks into our platform to then

26:43 make that available to our customers so that on day one anyone has you know world class go to market at their fingertips essentially. So, so that's that's that's what we're working toward

26:50 and that's that's the broader pitch. Um, we've built a version of an agent platform that allows you to to build a go to market specific agents, whether that's pipeline reviews, you know, on

27:00 the fly, whether that's uh TAM building on the fly, like TAM segmentation on the fly, um, you know, demand generation on the fly. So, like what kind of campaigns should you be running and how do you

27:09 actually optimize >> paid ad stuff? >> Uh, we we don't do paid ads, more like outbound related things. Um, so all the things that I'm sure you loved doing

27:17 when you were a sales person, spamming people, watch out >> today. Yeah. It's why you're so so good at building communities. You're so good at spamming. Um,

27:25 >> that's all it is. It's 90% of my job. [laughter] >> People are like this guy again. Or >> yes, I might as well respond. >> And then you Yeah, you get them on like

27:33 what is it? What is it like how many how many touch points? Like seven 12 before you give up or something. And and >> 12 nos. >> 12 nos. And so, but but you break people

27:42 down by like five or six. You know what I mean? Like, you don't need to get to 12. >> God, I am so done with this. All right. >> Yeah.

27:49 >> So, let that be a lesson to everybody that's listening. If some sales guy is hitting you up, just say no and then set up a reoccurring agent to say no every day and

28:02 uh you should be fine. >> Yeah. So, 12 nos and then they'll probably if they if they if they're reading the same book, they will they will

28:09 >> I [laughter] think that did come from like this guy Chad Holmes. >> Oh, that sounds familiar back in like 2009 or something.

28:18 >> Which one is this? >> Was like the sales machine. >> Oh, yeah. Yeah. I think I've actually I actually have read this book. Yeah. Yeah.

28:23 >> He used to go for like 15 nos. >> Yeah. Yeah. >> And he would tell people like until you get 15, don't even think about stopping. >> Yeah. Yeah. Yeah. Okay. So, so um look,

28:32 there's a >> to the to the recipient that might seem incredibly annoying. However, >> aggressive. However, okay, like >> yeah, aggress aggressive and and yeah,

28:42 just leave me the hell alone. But but having worked with our go to market folks, um seeing how they think about it is like pretty enlightening, right? It's never it's it's um it's not just like

28:54 bump bump like like you know, >> if it's no value, you're not adding anything. It's just, hey, I saw this in the paper. Thought you might want it. >> Totally. Totally. Totally. Exactly that.

29:04 And that stuff is like that that makes sense, right? I mean, that actually like if if if you treat sales as not a what do I stand to gain from you as the salesperson, but rather what what value

29:14 can I offer and be more how can I help you? I mean, you you know, you know this, you live this, right? Um, I I think that's the most effective kind of sales. And so a lot of, you know, our

29:23 our sales people do that and and then we try to capture exactly those kinds of insights in terms of how we help people do outbound and how we help people um get in front of their target buyer and

29:32 the target companies they want to reach out to. Um anyway, it's it's it's been a really fun product to build. Uh and >> and how does AI get incorporated in that? I imagine you're listening for

29:44 things and then you can craft a message. >> That's right. Is that right? Exactly. So So >> that's one way, but what else? >> Yeah. So we have a number of different

29:51 things at our disposal. So um everything is agentic of course. So we're we're getting signals from a number of different places in in the platform. I mean we're an all-in-one. So we we don't

30:00 just do like a CRM or you know just the outbound or just the data enrichment. Uh we've we've deliberately gone and built everything from the ground up. >> Oh wow.

30:08 >> So we are like the one-stop shop and because of that we get access to everything from you know emails that have been sent uh meeting recordings and transcripts, Slack messages. So all

30:17 these things that come in and then we can do you can use all that to infer um or level tone or or level understanding of a business context. So like what is if I am I don't know if I am Uber who am

30:28 I actually Uber is not a high rating level because it's like more consumerf facing but if I am if I'm a like if I'm a base 10 and I'm selling to you know >> because you just saw one of their

30:39 >> Yeah [laughter] like we're an SF people in case that wasn't clear drive by 10. [laughter] We have Andre Horowitz literally right here. >> There you go. I think we just passed

30:49 pylon. Uh yeah. So, so you know any of these B2B SAS companies uh >> there is a certain understanding about their ICP they have. There's a certain understanding of like their target uh

31:00 demographic, their target like size of company, the person at those companies they need to be targeting, the pain points totally 100% all that stuff. And so, so we can glean all that through the

31:11 collateral they upload to the platform or the stuff that we implicitly learn through sales interactions, through meeting transcripts, through emails. We also can glean that because we're seeing

31:19 them >> if B 10 is coming out with some kind of announcement or some PR, whatever it may be. >> Yeah.

31:26 >> And they're focusing on this. It's like, oh, hey. >> Yeah, that's right. >> Should mention something to your contact >> 100%. Yeah. And so so you know and

31:34 that's something stuff we capture through platform level interactions. And then we we have this concept of like Monaco memory which is effectively like a um kind of related to if you remember

31:44 Audre Kathy like many months ago brought this concept of like an LM wiki. Yeah. So like the LM wiki like we have it's like a version of that this um this persistent uh store of everything that's

31:56 happened and everything that we know about both the organization as well as the user at the organization. So your tone when you reach out to people, hey, when Demetrius reached us reaches out,

32:05 he's going to be super annoying. And it's like that's [laughter] >> one, you know, but that's your tone, right? That's like that's [laughter]

32:18 >> we can learn something like that. [laughter] >> Ouch. All right. >> Exactly. >> And I only need like two emails to glean

32:25 that. You know what I mean? I don't [laughter] >> All right. So, >> no. No. I have an agent superpower my skill augment my ability.

32:34 >> No, no, no. Jo is very effective what he does. Of course. Of course. >> Up to an 11. [laughter] >> But I didn't realize I didn't think about how many different insights and

32:44 actions you could take from having all that data. >> And the sales people that I know have cobbled that stuff together. >> That's right. That's right. in a way

32:51 that they're like they're recording with one software and then they have some kind of a automated pipeline where it will be >> put into their chatbt or claw and then

33:01 from that they get insights and it gets thrown into HubSpot and then that's exactly right they can write a report at the end of the month with all these calls that they've had with all the at

33:11 the end of the week whatever it may be when they have the pipeline review. >> That's correct. Is that correct? So now you're saying, "Oh, cool. We've got your pipeline review already ready because we

33:20 have all of this stuff and we've set that up." How much work is it on my side to put together >> all of these tasks or they're not even tasks, they're like

33:31 >> different lanes that I want. >> Very, very little, honestly. Uh, and part of it is because I mean the platform is robust on its own. But the other thing that we've leaned into a lot

33:40 from the get-go, which I think differentiates us from other products in in the category, is we are we we have we're actually like a little bit more hightouch on the human side than maybe

33:51 what traditionally so you know the PLG self-s serve like you just log in and you just start cranking you start adding rows to your tables and whatnot. It's it's that's not the experience here. The

34:02 experience for us is every single person has a designated what we call an FDA. So a forward deployed account executive. So it's effectively like your sales consultant um that sort of guides you

34:14 through the process because for a lot of a lot of our early customers, you know, we we have done really really well with like early stage founders that are all like very technically minded, very very

34:22 product minded, um but maybe don't have as much experience on the go to market side, right? And they're they're learning these they're kind of picking these skills up as they go. And so we're

34:30 like, hey, we've we have decades of experience doing this. So day one you come on board to Monaco and you have a built out TAM for you like 3 to 20,000 depending on your TAM size companies

34:42 that have been scored and ranked ready for you to go. Here are the buyers at those you know and here's how you can get to those buyers based on like LinkedIn connections and all these

34:49 things. All you have to do >> Yeah. Henry all these things. Yeah. And and and all you do is hit go and then and then sequencers getting sent getting followed up with you know you're getting

34:57 insights brought to your to the platform as as as things happen. >> Don't sleep on the handwritten notes. Those are actually quite useful. >> No, we we do that. We like I No. Yeah,

35:05 of course. Like Yeah. When I mean we uh we had handwritten notes. We had this campaign where we were giving, you know, Monaco. So we we do stuff in we have like a bit of a brand around gambling

35:16 and like the finer things and F1. And so we we had um we were one part of our campaigns was >> Taxision. [laughter] Whenever I think Monaco, I'm just like

35:26 >> no taxes. >> No no no none of that. None of that. Um none of that. None of that publicly. None of that on the record. >> Yes. Cut [laughter] that out.

35:34 >> Walls is going to be >> No, no, no, no, no. Anyway, you got the brand around >> You got the brand around the gambling. So, we actually had um we had uh we do a

35:41 few things that are kind of fun, like we we gifted people like poker sets, the >> poker night poker night. We do the poker night. Um that's happening, I guess, next week. Uh I don't know when this is

35:51 coming out, but this is October 29th. So, we're doing one that's I I think we our claim to fame on on that front is having the most um like the biggest prize, biggest cash pool on on poker,

36:03 $100,000 for the final table. So, it's uh >> you know, let me tell you, somebody walking in, I've seen this twice already now. Somebody walking in to just a

36:12 random like Thursday night poker game and like, hey, I'm going to eat some food, play some poker. And they're walking with $30,000. You're like, what? [laughter]

36:20 >> You're like, what? >> That is BC money. Well, >> there you go. There you go. >> You know, it's it's it's about brand activations or brand awareness, right?

36:29 >> That and I think you guys do that really well. >> I saw this stuff where you're flying planes over SF. >> It's fun

36:36 >> with the Monaco symbol. >> Yeah, [clears throat] >> I think just earlier today I saw somebody with a Monaco tote bag and so you're doing well on that side of

36:44 things. When we talk about the tech, >> Yeah. >> What are you actually using? Break down the stack where the harness is, all of that stuff. Yeah.

36:52 >> Totally. Um, so a lot of we we've hone grown our own harness, right? So a lot of so there there are pieces of the harness that we have used vendors. So for like versell sandboxes, right? So we

37:03 use them. We don't believe there's value in us building microVMs, you know, or like maintaining our own version of that. That that would that's what I would call undifferiated heavy lifting.

37:11 But but at its core, we do have our own harness. We think that's a good competitive advantage is to like re lean very heavily into the primitives around a go to go to market specific uh harness

37:21 including the skills >> that's the harness for the agents that the end user is using not your internal harness for coding and >> no no yeah not not the yeah not that

37:31 yeah exactly right so so good good distinction so we have we have our own internal coding system we've also built but this one the example that I'm giving is specifically around go to market the

37:40 ones that produce the go to market agents our customers get to benefit from and So you know at its core we've we've built effectively this like system that goes from natural language into this uh

37:49 like intermediate representation like you have you can think of like a JSON or sorts um you know JSON could be YAML could be whatever you want that represents like the workflow of the

37:57 agent and then that gets compiled down into like a lang graph essentially um a langraph graph uh that you can then run and and you know can loop and and do uh the kind of self-healing that that an

38:08 agent will do um you know and we can the underlying so effectively it's like a compiler like we go from natural language to compile to an underlying engine that is the the workflow builder,

38:17 but we could swap it out into any other workflow builder if we wanted. Um, and then the the the secret sauce a lot of the times is like what are we built around the memory? What are we built

38:25 around the tools that we've exposed compaction like how do we compact more intelligently? How do we you know all these things that are >> specific you the data that we access,

38:34 right? So, of course, we have very particular data characteristics because we're dealing with in some cases like >> thousands of meaning transcripts and hundreds of thousands of emails for some

38:43 of these orgs that are >> that was what I was thinking about is how you can avoid just bloat. >> Yeah. Yeah. It's uh it's there there there's a sort of tips and tricks around

38:53 this. Um but at its core, it's like you just have to be a little bit more intelligent around like the MCP tool MCP server tool design. Like you can't do you can't like I think the the the V1

39:03 vanilla way that a lot of people treat MCP is like here's an API here's a onetoone translation of the MCP tool for that API >> the worst way to do it. Yeah, it's not.

39:11 So, you have to think about it more in terms like the semantics of what is it the agent needs to do? Hey, I need to query meetings and you can do query query meetings either through uh like

39:20 keyword searches or through like a more semantic relevant search based thing. And so, you have to make like a single entry point for for that tool call for example and then allow both modes as as

39:29 as a way to you know query the actual data. So, um >> I've heard it said as doing the intent. >> Yeah. >> Like really designing your server around

39:38 intent. >> That's right. versus tools. >> That's exactly right. That's exactly right. Yeah. I think tools Yeah. The latter is like v1 APIs, right? This is

39:46 like 2000's level API tool definitions where you have like get ID. Well, I mean, you know, like like this is when we and not not to be derogatory, but but in like a uh that's how we started when

39:57 we were building web apps of the early 2000s is like, hey, get ID, you know, set ID, like all these get and post style uh HTTP methods that we defined. And now you're right like agents

40:08 actually can think at a higher level or like can operate at a higher level. I don't want to anthropomorphize your >> but you think that that is where your IP is happening like for you building this

40:18 product. Yeah. >> Which potentially can be >> headless completely in the future. >> Mhm. >> And people interact with Monaco solely

40:26 through the MCP. >> Yeah. >> Then your special sauce is how you're compacting how your memory is how your search

40:32 >> Yeah. >> functions work. >> Yeah. I think that's right. I think it's a combination of the fact that because we're the all-in-one, we are both the

40:38 the system of record and then what I I would call like the system of intelligence. So the system of record is we still are the place where you want to store like CRM level data, right? So the

40:47 the evolution of your deal pipeline, the accounts you've reached out to all these things. So, so that still is baked into the platform and then you need abstraction layers, you know, tools in a

40:56 lot of cases to interact with that data, reach from that data, write to that data and so you can do that either through like an external MCP or you can use that all internally and use our agents which

41:04 are baked into the platform. The the benefit of our agents are they're going to be highly optimized to the workflows that we care about. They're going to um the defaults are going to be they're

41:14 going to capture a lot of the the best best-in-class practices from our own go to market teams. like it's remarkably difficult to build like a really good TAM, right? If somebody comes and

41:23 they're like, "Hey, I'm selling to a marketplace, but like they're like think of them like Uber, but you know, like they're not too big." And so, but it's in like, you know, it's in like

41:33 >> East Asia. >> It's in East Asia and it's for shipping companies and you're like, "Okay, how many companies of those are there that you can

41:41 >> insert?" Yeah. Yeah. Yeah. Exactly. Right. Exactly. So like you know and so we've we've um you know the IP is like the data layer the context layer of course and then and the fact that like

41:52 >> all that is is resting on the bet that is like the system of record right >> and how much are you enabling the other software engineers at your company as your job versus building these

42:03 externally facing agents. >> It's both. It's both. Yeah. It's it's definitely both. Right. So so we have to do both. I think that the where we've really thrived as a company is domain

42:15 experts working handinhand with like subject uh with like software engineers right that can like codify the workflows of the of the domain experts and so in practice this the software engineers

42:26 have to be empowered to build very quickly to ship really quickly to to have the primitives to to build on top of the harness and then the subject matter experts need to be empowered to

42:36 you know know how to express their intent as well and the things they care about and then because the the the failure mode is like the translation, right? Is like how if technical person

42:46 talking to nontechnical person it sometimes they can like kind of miss each other or not really not speaking the same language. >> Um

42:53 >> well dude we're arriving at our destination I got to kick you out. >> This is fun. Oh gosh. >> As much as I want to keep talking. I'll see you on Thursday though.

43:02 >> Yeah. Yeah. Yeah. >> You are a person of interest in the magazine we created. >> Uh oh. Uh oh. That is That sounds like [laughter] It sounds like I'm I'm being

43:10 sales emails. >> Well, that or or it sounds like I'm being tracked by the FBI. [laughter] Like he's on a wanted list somewhere. I I'm cool. I'm cool.

43:20 >> I'm cool. Whimo people, don't worry about me. >> Yeah. Hey, Who. Don't >> don't need to record this. >> Don't need to record. Lock the doors.

43:29 Don't let them out. >> Yeah. But this has been cool, man. >> It's fun, man. >> I appreciate you doing this. >> This is a cool concept. I think this is

43:34 this is a I'm a big fan of this. Hey man, it's so fun to see you, man. I'll see you on Thursday. Thursday. Yeah.