How Forward Deployed Engineering is done at Ramp — Leo Mehr

[music] >> Awesome. Thank you, guys. Awesome. It's great to to meet everyone. I mean, I hope that after the talk, you know, if you want to come out and we can chat, we'd love to. Um Cool. So, yeah, today my goal is to share with you guys the two most important principles from what we learned doing FDE at Ramp. So, just yeah, briefly a little bit about myself. Yeah, I'm a director of engineering at Ramp. Uh I joined the company 2 and 1/2 years ago when it was just you know, FDE was just two engineers at the time. And today, my org is about 30 engineers across four deployed developer API and our new AI services um business. So, I know this is kind of a running theme, but like no one knows what FDE is. So, I'm just going to spend a moment on that. So, yeah, I I I actually kind of like this meme. It's To me, it's kind of funny. Um I um but I I I actually think it's like totally wrong. I don't see this as the actual like true form of what FDE is. Um I don't see it as like the final evolution or like boss mode of technical go-to-market roles. Um now, this might be true at some companies, but at least at Ramp, it's a little bit different. So, FDE at Ramp, uh we live within the engineering organization. And our goal is to help Ramp win upmarket. So, with that in mind, what we do is we basically work on the core product and our new agentic features and make them work really well for our largest enterprise customers. So, that's just a little bit of intro context. I want to dig in and and today, like I said, there's just two things I'm going to share with you. Literally two things. Very easy talk. And these are the principles that I would say have really guided us and I would say probably the two most important things that we have. Always be scoping and scale with tokens. So, let's get with start with the first one. On scoping. So, I would say there's this thing where like people many many people think that as an FDE, your job is to just say yes to the customer. But that's wrong. If you were just to say yes, you know, instead of like beautiful Waymos that we have driving us around in San Francisco, you'd have something like this, you know. Yeah, horses with like rockets strapped to their legs. And the point is you you want to help the customer be successful. You want to try to figure out a way to say yes. But you actually want to deliver good software. You need to build the right thing. So, you don't just endlessly say yes to people. And I I do want to share an example of something that I would say happens somewhat regularly in one form or another at Ramp. So, it's Friday night and an enterprise sales rep comes to us with an urgent request that this super important strategic logo is only going to close if we build out an SAP S/4HANA integration. And I think that the default engineering reflex is like, "Shit. Like, where are the SAP API docs? Like, where do I find them and how do I build this integration?" But what an what a well-trained FDE would do is like pause for a second and say, "Okay. First of all, like, what's driving the urgency here?" Like, one thing I've seen is I've seen sales reps who like go kind of crazy because it's like the end of the quarter and they're trying to hit their quota and close the deal and not because the customer is the one driving the urgency. So, that's like one example. >> [clears throat] >> But, you know, as an FDE, you're asking tons of questions to gather context about what's important um and what actually is the right thing to build. And so, you might ask like, "Who's using this integration? Have we exhausted all the different workarounds? Is there something manual that we can do in the meantime? Does the customer have like technical resources? Can they hit our API such that we don't have to build this thing?" But, I'd say the most important thing that an FDE does is also looks beyond this one request and looks at the other prospects that are coming down the pipeline and other customers to see if anyone else would benefit from this as well. And the point is that by gaining all this context, you can do a better job of building the right thing. So, I want to share another story that was really really painful for us in the early days. We had this large enterprise customer and they needed this reimbursement feature on mobile. Unfortunately, our mobile team was totally swamped. So, we basically just had to roll up our sleeves as FDEs and just get find out a way to get things done. And we had two of the engineers on the team just like learn how to do iOS and Android development. And it was awesome. We were super excited. We're like, "Okay, we're going to ship this feature. It's going to be so good. Like, hell yeah." So, we grinded for a couple weeks, got the feature done on both platforms. And we go to the customer and we're like, "Awesome. Like, can you send us your list of, you know, beta users for Android?" And that's when they told us they only they they they require they mandate all of their employees to use iOS devices. So, you're like, what the [ __ ] Like Not not to the customer, you know, just internally. But like obviously it was super disappointing for us because we'd put all this effort in. And so, it was a a big lesson for us to remember the importance of scoping. Even some of the most basic assumptions like which you know mobile platform you build on it's it's super important um to validate them and and thus kind of emphasizes the importance of scoping up front. Now, Okay, so let's say that you and your team have become masters of scoping. You know, you're you're amazing. In today's world, this is not enough. So, unless you are scaling with model capabilities, you are going to fall behind. Now, I'm not going to belabor this point too much. I think like every talk in this uh in in this conference is probably some flavor of this, but like the point is that we basically have to reinvent our jobs constantly now. So, whatever work we are doing today, you know, for the most part it's knowledge work, we have to figure out how to have models and agents do it for us. And so, that brings me to the second half of this talk and the the other point that I want to convey today, which is all of us have to figure out how to scale with tokens. And the way that I interpret scaling with tokens for FTE is take a look at the whole life cycle of what an FTE does. From gathering context to scoping out a request to writing out a spec and then implementing the feature, each stage of that pipeline can be replaced with agents. And at first it seems kind of daunting. You're like, like how are you going to go and approach and like solve that? But if you break the problem down and then make progress on it, it's it's actually pretty tractable. And so, I'll share share with you guys one example of something Oops. Something that we um that we've done at Ramp. So, we have this internal Slack channel called FDE requests, and this is where account managers, solutions, uh sales reps will post whenever there is a blocker for a prospect or customer that's large enough, basically. And so, we get these requests. In this case, actually uh one of the CSMs on our team, Greg, posted here. And um if you were to It's actually a Notion workflow. If any of you work at Notion, by the way, thank you. We like use Notion so much. Um if you were to click open in Notion, you'd see like a pretty long request that has all the details of what what exactly it is. And the problem is there's a super high variance. Like, some people will submit like really detailed, good requests from the customer. And others are just going to submit like one line, like, "We need uh you know, we need this SAP integration." And before what would happen is we would have FDEs manually kind of go through this request. We'd read the whole thing, understand it, figure out what exists in the product, do a bunch of back and forth with the customer. And this is like exactly what the first half of the talk was about, always be scoping. You know, we would spend a lot of time really digging in and validating what exactly was uh you know, absolutely necessary. And so, you can see here what we what we did then was we basically um used Notion uh Notion agents to build a V1, which literally just took the request and asked a couple of questions. That was it. And um after It was kind of astonishing. Literally, after a couple of weeks, we found that it was like saving us a lot of time because, first of all, immediate like the latency of replies went from like hours or days to like, you know, seconds. And immediately, like the account reps that the account managers, the reps would start kind of engaging with this agent. And one of the things that we did was because it went so well, this this is actually um the more recent iteration of it. It's very cute, you know, the little penguin actually helps make it seem a little more friendly and approachable. Um and what it does is it actually goes and does several rounds of back and forth questioning with the submitter until it deems that it's ready to create a lot a spec, basically. And it's actually been incredible how helpful this has been for us. I I would say it's probably saved us like a large percentage, I don't know, 20% of the time that we'd spend on scoping out these requests. So, you know, this is this is a great example. For us, this has been really helpful. It's I'm super excited about this. It's going to help automate a lot of the work that we've been doing manually. But, um it's really just the first stage of this pipeline that I was alluding to. So, if you look at the first part here, like we've been able to make some progress on it. The last step as well, going from a a well-shaped spec to like a working product, obviously like Frontier models can like one-shot medium-size features. And so, the last part is also is is a lot easier for us. It's this middle part that I would say is super like gnarly and like unformed and difficult. And I'm I'm really excited about our team kind of investing a lot more and spending a lot more of our time just like building out this factory, building out agents to replace each one of these steps. And the thing is, if you look at if I were to say 6 to 12 months from now, like what does FDE at Ramp look like? Like these are the sorts of applied AI problems that we're going to be spending all of our time on, I think. Like, you know, making sure that the agent harness that's running each of those steps is running super smoothly. Um making sure that the the output quality of each of the outputs of the the pipeline is is actually good. That you know, with with evals, with rubrics, with human feedback. Um and there's of course like one of the biggest challenges, which is getting your agent the right context, you know, when you're making the alarm call, ensuring that it has the right context. So, there's like a lot of historical data, data about the the product. Imagine like all the knowledge that a product manager has in their head about their product. Like, how do you get that into an agent? Like, Notion docs and all your existing knowledge base and help articles only give you so much of that. Um yeah, skills, memories, tools. I could go on for a bit, but ultimately the most important thing here is that as an FD, we still have the responsibility of taste and judgment over the final output. So, that's going to be like the underlying kind of throughline. Okay. So, let's say that you've done an amazing job building out this factory. But the problem isn't to tie this to the first half of the talk. If you don't do a good job of scoping out requests or or building upon the principles of scoping things well, you're going to get a token maxing slop cannon. And so, the whole point is that you have to do these both because the other way around is actually quite bad as well. If you are, you know, amazing at scoping, but don't invest in building out this, you know, agent factory, you know, it's going to be over for you. Like, uh your your agent native competitors are just going to overtake you and outcompete. And so, that's why um in the end here, I want to close with the the the most important thing is that if you have both of these, it can set you up for success in the future. Always be scoping and scaling with tokens. The future of FD needs both. That's all. Thank you, guys. >> [applause] [music]