How do you diffuse AI into the real world? — Varun Shenoy, Long Lake

0:01 [music] >> Hi everyone. I'm Varun. I'm one of the co-founders at Long Lake and I'm excited to share a little bit about what we've been up to for the last 2 years.

0:21 It all comes back to a question all of us have asked time and time again. The models are getting better, but the real question is how do you actually deploy the AI into the real

0:34 world? How do you get the models to complete economically relevant tasks? Let me start by saying everyone has seen the demo. Think of the agent

0:47 automatically booking a flight, the agent automatically completing a ticket in some kind of customer service portal. Think of an agent completing a block of code ready to commit and go.

1:00 The reality is we've all seen this and it feels like magic. 2 years ago any of this would have been complete science fiction. The capabilities are real. Now, walk with me into a 200-person

1:15 property management firm. Real people, real properties, real dollars, real customers all across the US. You would expect AI to show up by now,

1:27 but the reality is nothing has changed at all. Here's the thing. This is totally normal and maybe in fact I'd argue this is what we should expect.

1:42 This is true for every general-purpose technology. You know, take electricity for example. Electricity was invented in the 1880s and it was first demoed at Edison's

1:53 Pearl Street Station Dynamo Room over in Manhattan. This was the magic demo of its time. The reality is it took a long time for electricity to be fully adopted.

2:07 Consider a Ford factory. It's not enough to just have electricity. You have to rip out the existing motors and equipment. You have to bring in the new equipment. You have

2:17 to go and train everybody to use that very same equipment. Here's a picture of a Ford electrified moving assembly in 1924. These things take time.

2:28 Diffusion of any technology takes a generation. And since everyone here in this room today is talking about AI, I would argue AI diffusion is perhaps the single most

2:40 important problem for the next 20 years. The models are going to keep getting better. The big question is how do we actually get these models to be in the real world, complete real tasks, uh and

2:52 make people more efficient, happier, and provide better service. So taking a quick step step back, who are we? Uh we are Long Lake. Over the last 2 years, we've raised over $3

3:04 billion from Elad Gil, General Catalyst, and AlphaWave since our founding. Here's the strange part. We we don't sell software. We actually go out and acquire and partner with real services

3:16 businesses in the world. Uh we've acquired 35 businesses across HOA and property management, architecture, HR services, and a lot more. To give you a little bit more flavor, we

3:28 have roughly a 40% team right now split between technology, finance, and operations. More than half our team is part of the technology team focused on uh building products, data, and

3:39 deploying the core products into the field. Uh we're in a collected group of folks, a bunch of ex-founders who've worked in the services before, ex-military, folks from Palantir, Ramp,

3:50 Glean, uh and from the finance side, Blackstone, H.I.G., et cetera. We are we are not selling them to these companies above from the outside. We're actually deploying into these companies and

4:02 figuring out how to get the technology to work. And just to show you the scale we're playing at, we announced recently our $6.3 billion take private of American

4:11 Express Global Business Travel, the world's largest corporate travel platform. We own these businesses. So, when the AI doesn't work, it's not

4:22 their problem. We're not the vendor. It's our problem. Concretely, again, we are not the vendor. We are the operator owners. And we work very closely with our teams

4:33 within the businesses to drive real outcomes. Now, I want to step back and get to the concrete about the how. What are the lessons we've learned over the last 2

4:43 and 1/2 years? And what we've learned from deploying AI into companies we've owned. Three quick lessons. One, how we move agents from co-pilots to co-workers.

4:57 Two, how we leverage real-world data within these businesses. Remember, we're seeing all of the work that's being done in these real services businesses. There's a lot of interesting

5:08 problems and solutions embedded within that. And then finally, perhaps the most interesting and exciting is how do you actually get all of this technology to

5:16 compound over time by learning loops in the enterprise. We'll get to that at the end over here. So, starting off from co-pilots to co-workers.

5:27 There's a spectrum of how much autonomy you can give an agent. On the left here, you see a co-pilot. This is, you know, your simple rag chatbot from 2 years ago. It's very

5:36 quick. You can ask a question. Maybe it's integrated with some systems. It can give you information back very, very quickly. The second step is a synchronous agent.

5:45 Consider something like Claude code, Codex, Claude co-work. It's real-time. There's this two-way interaction. It's a bit more sophisticated than a co-pilot. You can go let it run off for 1 to 5

5:56 minutes. Uh it'll call tools, maybe use its skills. Uh it's still synchronous. You still need to step in and ask a query. So, the next obvious rung of the ladder is the asynchronous agent.

6:06 You can come in here, still ask a query. The agent will go off into the background, do some work, and then come back. Uh and what's really interesting about asynchronous agents is that the

6:15 user does not have to be the one that triggers them. You can have external triggers as well. Maybe someone completes a certain task and there is an async job queue uh that

6:25 allows the async agent to pull off from and proactively offer advice to the end user. Then, I'd argue the next step is a long-running agent.

6:35 How do you get these agents to work for hours, days, weeks, months, etc.? I think this is currently a very core problem that a lot of the labs are focused on, as are we.

6:48 And then finally, at the end, the holy grail, an AI co-worker. This is where most people start off. You want a proactive partner that gets work done just alongside you.

7:00 This is what everyone wants to sell you, but what we've learned from owning the outcomes in this business is you have to earn the right to do more. It's it's not enough to jump to the

7:11 co-worker immediately, right? For for a bunch of reasons. One, for certain tasks, the models might not quite be there yet. And two, you actually have to work with these

7:20 companies in the field, interact and iterate very, very closely, so that they understand that this is the beginning of AI, and you can work up the rungs over time.

7:33 I think a really unique lens to look at this problem through is the that of the jagged frontier. We all know that agents are incredibly good at writing code. So, what does the, for example, synchronous

7:44 agent for code generation look like? This is super simple. This is just your coding agent, maybe it's Codex, Cloud Code, just running on your desktop. It has access to a file system. You

7:54 collaborate within real time. You get instant feedback and you iterate. The next step is, you know, if you look at code code generation, what is the async agent? This is also fairly

8:04 straightforward and largely solved. You take the exact same coding agent, you wrap it in a sandbox, and you just let it go run. It can build, it can test, and once it's done with its work, it can

8:14 provide the code in the form of a PR. One thing that's really unique about engineers is folks are incredibly good at already paralyzing their work. It's very commonplace to launch 10 jobs

8:26 and be comfortable with the fact that job seven might finish before job three. So, engineers are incredibly good at using these async agents. Now, when we come to services, the

8:37 equivalent of a synchronous agent, what we talked about a little bit earlier, it's a co-working agent. It's an agent that has deep context about your enterprise. It interacts potentially

8:45 with MCPs, custom tools, custom integrations, uh and you can chat with it synchronously just like any of these other products. I think this is a frontier here in the

8:54 bottom right. What does it mean to build an asynchronous agent for the services? What does it mean to paralyze work in industries where work is traditionally

9:05 done in a very, very serial manner? This is where we spend a lot of time and this is what I wake up every morning really excited thinking about, you know, we've we've figured out what the async

9:15 and forking mechanism for code is. You just spin up a bunch of sandboxes and do work. What does that look like for the rest of the world? So, here's a couple questions we think

9:26 about pretty seriously. One, you know, the models are trained on code, they want to write code, they're incredibly good at writing code. How do we leverage these coding agents for actual knowledge

9:34 work? You You rather than wait for the models to catch up on doing services knowledge work, what if we just use that code knowledge and represent knowledge work as code?

9:44 Two, as I mentioned, engineers are used to paralyzing work. How do you paralyze work that's traditionally serial? You know, people clean out their inbox one email by one email, not 10 emails at

9:52 once. And finally, how do you move up the ladder here both in terms of product and user enablement? What are the right form factors? And I'd

10:01 argue this varies dramatically from industry to industry. Just because you have one way of launching an async agent for code, doesn't mean that same way is going to work for architecture or

10:10 property management. The second point I want to cover today is leveraging real-world data. We all know this. Frontier models have learned from everything humanity has

10:21 written down, but the most valuable tasks are not on the internet. How do you actually close the books when you're missing receipts?

10:29 >> [snorts] >> How do you scope a building for construction in a blueprint, potentially collaboratively? How do you coordinate vendors for fixing

10:36 a broken roof? All of this knowledge lives in people's heads, in 20-year-old software, uh in the way that one senior person on one of these teams just knows how to do it. How

10:48 do you make this information explicit and create tasks that you can actually learn from? So, we've constructed a little bit of a flywheel. We get our agents to

10:56 collaborate with our employees to do real work. And this allows us to generate rich traces of data and information. Tool calls, the hiccups, the papercuts, everything that goes

11:06 wrong with doing real work. This in turn allows us to build real-world evals. There is a ground truth here. In the case of the roofing example, the

11:16 question is, did the roof get repaired? Did the books get closed? And this allows us to hill climb and build better agents, which leads to more and more impact. And what's really

11:27 exciting is it ratchets up. Every week our hill climbing benchmarks become a regression test. So, our agents get better and better over time. Just to drive a little bit deeper here

11:39 on the traces, there's three upshots of being able to collect these rich traces. One, we get to generate amazing evals that are built and scored automatically. Uh and we're able to gather both

11:51 implicit and explicit feedback. Explicit feedback in the sense of thumbs ups and thumbs down, maybe people provide a note telling us whether this response was good or not. Uh and also implicit

12:00 feedback. Right? Again, we have the ground truth. Maybe there's some data that the AI generated and there's a real diff between the data that the AI generated and what was ultimately

12:09 submitted. That's rich information that almost no one else has. Two, we've started post training models internally on all of the data that these businesses

12:19 operate on and produce, generally speaking. This is all data that is completely out of distribution for most frontier labs. Think of the task I showed at the

12:28 beginning. A lot of the models A lot of the frontier models today just can't do these tasks yet and we're trying to post train our own models internally to be able to do that on the rich source of

12:38 data that we own. And then finally, the actual agents themselves. The real world is incredibly hairy and messy and you want customization per

12:47 company. Every company does things very differently. Customization per user. The way each user does their work is very unique. And customization per client. The way you work with every client is

12:59 different. It's a services business and you want to uphold those standards. I love this picture because it's the whole thing in a single image. Um the the way we usually talk

13:10 about LLM tasks is the top panel. Right? You just It's It's a slope. You got a bike. And but there's clear sight to success. The reality is most work is not like

13:22 that. And And you and I both know that. Uh there are hills and ravines. Uh there's death by a thousand paper cuts. But But that's what real work looks like. That's the entire job. The

13:33 exceptions are the job. That's That's the demo. That's the actual job. Now, on to the final thing I want to chat with you guys today is learning

13:47 loops within the enterprise. I'd argue there's two hot trends everyone's talking about in 2026. One, it's continual learning. How do you make an agent better over time with feedback?

13:58 I think there are plenty of sessions uh this week on how you can use continual learning, whether it's in the prompt or in the weights. And two, enablement. How do you get in

14:08 these enterprises and actually get them to adopt and use AI? Traditionally speaking, these two initiatives are owned by two separate teams. Right? The continual

14:18 learning is owned by your research team, your platform engineering team. Enablement's owned by growth or deployment or customer experience. Uh usually pretty siloed, not much

14:26 interaction between the two. We think these are part of the exact same loop. The agent only improves if people actually use it.

14:37 And people only use the agent if it's worth adopting. So, here's a little graphic of a snowball. More usage drives continual learning, which drives a better agent,

14:46 which drives more usage again. All this to say, there's still a really big elephant in the room. How do you get the initial usage? I think a lot of people, you know, will

14:57 use Claude Code or or give it to their whole enterprise, expect folks to just start using it. Everyone assumes the usage just shows up.

15:08 But as we all know, that's simply not the case. It never does. Right? Getting a 100-year-old firm to change its processes is hard. You could have the best AI coworker on

15:20 the internet or on Earth. And if the people if the person who's closed the books for the last 20 years continues to do things the same way, nothing changes. Nothing happens.

15:32 So, what can you actually do about it? What you know, this this seems like incredibly hard. What what's the upshot? How do you actually get this stuff to work? Well, I think a lot about Jensen

15:41 and how he dominated the market in his words with extreme hardware software co-design. Designing the chips and the software together as one system. We look at this through the lens of

15:54 extreme software service co-design. How do you co-design our products with the people and the processes at our businesses? And I'd argue this is only possible from being within under the

16:06 same roof. We need to meet the people within these companies both metaphorically, for example, bringing products to their systems so that the energy required for

16:17 enablement is kept low, and also physically. Get on a plane, show up, say hi, learn what people actually do. You know, maybe you build a product that's natively embedded into Excel or

16:29 into their ERP system, maybe their 3D design software, or or maybe even their Microsoft products like Outlook, Gmail, etc. Or you show up in person. You do a lunch

16:39 and learn with a bunch of folks at one of the companies. You go to their conferences and you create cotton candy and run a stand for them. You go mountain biking and ask them about all

16:47 the difficulties that they have with their actual day-to-day jobs. Or you show up in person one-on-one or sometimes even two-on-one in this case and just show them how to use the tools

16:57 and learn from the feedback because this is what the rest of the world really looks like. It's not like the folks in this room or in San Francisco. It's a lot more like this. You cannot co-design

17:07 software with the services business over Zoom or over a support ticket. You you have to be there. You have to be in person. And I'd argue this is the part that

17:16 actually makes it work. In order to get AI diffusion to work, you have to touch some grass. Thank you so much. I'll be around for the rest of day if there's anything I

17:24 can help with. My email is up there. And yeah, thank you. >> [music]