Does splitting the prompt cut token use in half?

0:00 Every voice agent framework or platform today has its own definition of what an agent is and what it can do. Build on ElevenLabs, Vapi, Retail, or

0:10 Balna, and your agent stays there. On the open frameworks, Pipe Chat and LiveKit, [music] a Pipe Chat agent can't easily be ported

0:18 over to LiveKit and vice versa. You pick a stack, you build inside it, and your voice IP [music] is embedded to whichever one you started with.

0:28 This changes today. Voice agents [music] are a primary customer interface, but most companies don't actually own the IP

0:37 >> [music] >> that they're building. It is embedded in third-party platforms. On many platforms, an agent's behavior is scattered across prompts, framework

0:46 code, provider integrations, and runtime decisions. Change the stack and you rebuild it. Change the model and its behavior [music] shifts.

0:55 Change your compliance rule and you have to update every deployment. That's why we built Unmute. I'm Luke, founder of Slang, and today we're launching Unmute as an open

1:06 declarative standard for voice agents. At Slang, we run voice infrastructure across 15 sovereign regions, primarily for regulated customers. That work has shown us that three common

1:16 [music] assumptions about voice agents are wrong. First, voice agents are not a sequential turn-taking pipeline. They are concurrent, multimodal, real-time system.

1:27 Listening, reasoning, retrieving, speaking, and interrupting can [music] happen at once. So, the question isn't which model do we use across the sequence? It is which

1:36 resource should the agent use at any point in time? The resource could be a frontier model, cached audio, a compliance-approved disclosure,

1:45 or available compute in the right region. The agent should compose those resources around the caller, the use case, [music] and the rules, without being rewritten

1:55 when the infrastructure changes. Second, today's voice agents assume the LLM should be the decision-making spine of the call. Tool calls, data [music] flow, step

2:04 transitions, failure handling, all left to the LLM runtime. Unpredictable, expensive, and hard to govern. With Unmute, declared execution runs the

2:15 call. Tool calls, data flow, step transitions, failure handling, all declared, all enforced by the compiled project. The LLM is called as an optional

2:25 resource, which earns its place. Which means one package compiles to Pipe chat cloud, LiveKit cloud, or Slang. Cascade or speech-to-speech architectures

2:36 in the region that you choose. Third, today's voice agents assume you have to choose between rigid IVR style on one end, or a detailed prompt followed by hope on the other.

2:49 Two positions, both brittle at scale. Great demos fail in production is really this false choice biting. We never specified [music] what the agent was supposed to do.

3:02 With Unmute, determinism is a dial set turn by turn. Lock a regulated disclosure, leave an open-ended objection generative. Same call, different levels of control.

3:15 Non-determinism becomes a resource, not the foundation you spend the rest of the system trying to contain. That's what Unmute makes possible. [music] The agent becomes the artifact. Its

3:27 behavior, its resources, its integrations, its compliant boundaries, all expressed in one portable package. Everything, the framework, the runtime, the cloud, the region, is

3:37 infrastructure. Compliance boundaries live in the package. Update them once, recompile. Every deployment inherits. Unmute compiles to Pipe chat and LiveKit

3:47 frameworks. We have been inspired by both, and my learnings from 5 years at Vercel, standards outlive platforms. Nico will now give you a more detailed

3:56 introduction to what that looks like. >> I've been building agents at Slang for a few years now, spending [music] nearly all my time with customers and in open source. And what I see is always the

4:10 same problem. Everyone builds [music] an agent the same way, one prompt. Every time they need something new, they just add it at the bottom of it. And the demo looks great, but then it goes into

4:21 production and [music] it breaks. And the reason is always the same, the prompt. Because the agent reads every rule at every turn. That is just too much. Goes off topic, [music] picks some

4:31 rules and ignore the others. And when something goes wrong, nobody knows which part of it is causing the issue. Because nobody ever tried to mentally break down the prompt into the steps that the agent

4:42 should go through. What I learned is that if you want agents that work at scale, [music] you go towards determinism. Not by taking decisions away from the model, but by giving it

4:53 less to decide at a at a time. Split the work into steps, give each [music] step only the context it needs. At Slang, we have been looking for months at where the time goes at each

5:04 turn and where the mistakes [music] come from. Both are in the same place, the LLM. So the question we work on is simple. How much can we take out of the LLM and still keep [music] what makes it

5:15 good? And here is where Ammute comes. Ammute is a declarative standard for building [music] voice agents. So you only think about the parts that really matter, the steps the agent

5:24 should take, what each step can see, and what runs before the call, how long the agent [music] can wait, and where are the models living. You write that down once and we build it for you. We map it

5:35 to LiveKit, Pipe Cut, or we deploy it to Slang and the good practices comes with it. >> [music] >> Here we have one agent for a salon,

5:45 built two times. Same [music] tools, same voice, same phone numbers, same framework under it. The only difference is how the work is arranged. On the left side, we have one agent, one prompt, and

5:57 every tool it may ever need, 10 of them. [music] And the model reads all 10 descriptions at every turn. And the prompt has all of it encoded, all the rules, [music] all the edge

6:08 cases that came out from real calls, check who's calling, take a booking, handle complaint, answer questions about the salon. >> [music]

6:15 >> There is a section in which whose only job is to tell the model which of the four steps are we at. That's the deal. Four jobs [music] in one prompt. All four go to the model at

6:26 every turn when the caller only needs one. [music] On the right side, instead, we have the same salon, but with a split. We have the concierge agent, and we have the complaint specialist.

6:37 >> [music] >> The concierge is the one that agreed talks to the caller, has one tool, look up salon info, that is the tool that we only need.

6:45 Everything else [music] is delegated. And when we delegate, we drop the context and keep only the context that the steps actually need. And that is the whole declaration, what a step is, when

6:56 to run it, and what [music] it gives back. Here we see the verify customer step that confirms who are we talking to and hands to the manage booking the customer

7:05 phone number required for the booking step. [music] On the left, the confirmation is one line encoding to the prompt, confirm who are you talking

7:14 to before you book. You write it, and you just hope that it actually happens. On the right side, instead, the booking step is not given to the model until the number actually exists. So, a booking

7:27 without a confirmation number is not a likely, it's just impossible. And if we look at the prompt, >> [music] >> the entry prompt on the right, against

7:34 the whole prompt on the left, we drop about 50%. What we did is just split it. [music] This is the customer prompt, the the the hand hand prompter, [music]

7:45 a booking prompter, and a complaint specialist that is completely segregated from the rest. >> [music] >> And only one part repeats everywhere,

7:52 the voice contract, how the agent speaks, how it sounds, what it never says. >> [music] >> That part has to be true on every turn.

7:59 So, it lives inside the prompter. Everything [music] else stays inside the step that needs it. The booking prompt knows how to take the booking. It does not know the refund

8:09 policy, and [music] it does not need to. So, what we will do now is compare the same conversation live. One single prompter here on the left, one broken down [music] into steps, and

8:20 see what's changing behind the scenes. In real life, what you perceive is the skip of a turn. Here on [music] the left, the agent is asking the phone number, for example. On the right,

8:30 instead, the agent [music] already has it because we pre-fetched it from the carrier before the greeting. So, we skip [music] the whole turn. But in practice, when you look at the

8:40 numbers, and here is Lang fuels, one of the providers we use to measure how the agent perform [music] across different logics. On the right one, we see the salon

8:50 concierge broken down into steps. [music] On the left, the salon concierge as a single prompt. And the number of steps stays [music] more or less the same, but the token

9:01 usage is drastically higher on the left, the one single prompter, more than double. And that is because the entire [music] prompt is passed to the model at every turn, every time.

9:12 Whereas on the right, we [music] start at the entry point, we go down into the verification step, and we only pass the verification step into the prompt. Same conversation both times,

9:23 >> [music] >> less token, fewer turns, and lower latency. And here's the part worth saying out loud. Put the four prompts together,

9:31 and you get more [music] text than the single prompt on the left, nearly double. But this is not the number that matters. The prompt [music] is sent again on every single turn. So, what you

9:41 actually pay for is what sits in front of the model right now. And someone asking what time [music] we we open should not pay for the phone policy.

9:53 Here is something that we see nearly [music] every agent that we look at. For first 30 seconds of the call, [music] and here we can see the agent is asking things that could have known

10:03 before picking up. [music] What is your appointment? What is your number? Are you a customer or not? Each one of these is a turn, and a turn with a tool is two model calls plus the tool call, not one.

10:16 So, this [music] entire block that we see here run all the free pre-fetches before the greeting. Read the clock, take the number the carrier already gave us.

10:26 >> [music] >> Look at the number in our own system, and by the time that the agent says hello, it already knows the date, the number, [music]

10:32 and the name on the record. There are two matters. The first one is the one that people miss because a date tool looks like a real tool. It is not. [music]

10:41 A tool that takes no argument can only give you something you could have written in the prompt yourself. But, now let's go to the interesting [music] part, the second half, the one

10:51 that I find more interesting. Those are the variables, three of them. Each one declared once [music] with a type and a default. Type variables.

11:00 And here is the thing about [music] language models. It remembers nothing between turns, >> [music] >> nothing at all. Every turn you send is a

11:08 prompt, the whole prompt again, and the whole conversation with new messages appended. So, what you're [music] putting in front of it is not a detail, is the cost of the call, paid on [music]

11:18 every single turn. So, we let you declare where each value is allowed to go. Take the phone number, for example, that we see in here. It [music] tells you which phone code not

11:30 withholding it. So, somebody has to agree to [music] it first. That is the the the confirmed point that we see in this step. And it means [music] the number can

11:41 appear exactly in one prompt. The step that reads it back, not the greeting, not the [music] booking step, nowhere else. If I paste the number into the greeting, I get a build error. Most

11:51 importantly, when a tool needs that number, it does not ask [music] the model for it. We inject it there. The model never types the number, [music] so it cannot type in a shape that does not

12:01 match what the record we found. The idea is that if a step [music] does not need a variable, you just don't declare it. If you need it, you say so. Now, here's the whole point. I'm not

12:15 going to design this [music] agent. I'm going to describe the problem in one paragraph and let the coding agent build it. I'm going to install the new skill.

12:27 And that's it. The skill we just [music] installed give it everything we need to know about Ammute, so it can go and make decisions by itself. Let me start up Cloud Code.

12:41 The prompt. And [music] it works. Look at what it did. Figure out the right structure for us, and that's exactly what you want. Now, let's

12:57 actually run this thing. First up, we do the compile. Ammute compile. Compile takes the package and turns into [music] running code. Checks the

13:08 agent.yaml, wires up the tools, and makes sure that the whole [music] thing is actually valid before we ever run it. And now dev. Devs up [music] locally the agent, so I

13:20 can just talk to it. Let's call it. >> Thanks for calling Diamond. This is Ruby. What can I do for you? >> Hey Ruby, how are you doing? One paragraph in, working voice agent now.

13:33 That's Unmute. >> [music] >> This is four moves, but in reality, one shift. Define the agent [music] once, its

13:43 behavior, tools, integrations, and policies, then compile it to the runtime you choose. Select your infrastructure, >> [music]

13:50 >> either your own, Pipe Cut Cloud, Live Kick Cloud, or Slang. Cascade or speech-to-speech [music] architectures. Run it in the region that you need.

14:00 When your stack changes, your agent doesn't have to. That's the point. The agents remain your version controlled IP, even when the infrastructure changes.

14:10 Today, [music] we're releasing the Unmute standard under an MIT license, alongside the open-source CLI, and its first three infrastructure

14:19 targets, Pipe Cut Cloud, Live Kick Cloud, and Slang. Unmute is open. Take it, [music] build with it, and tell us what we missed.