Stateless, Yet Durable: MCP Tasks v2

0:01 [music] [music] Next up, going to bring on Cornelia. See, Cornelia, start sharing your video here.

0:18 >> That's up to me, not up to you. >> Absolutely. Great to have you. Developer advocate at Temporal. Uh, super exciting. >> Yeah. Thank you so much. And I mean,

0:30 it's perfect timing. So, Vicram talked about um MCP tasks just a little bit ago. I'm going to talk about them some more. And uh I was sitting there listening to Alex and I was like

0:40 peppering like when is Goose going to handle MCP tasks? And he answered the question before you asked it. Yeah, I've been working with with tasks and I I have implemented the client side of the

0:50 task protocol as well as the server side. And so that's what I want to share today. I haven't implemented it in any like standard client or anything like that but I have my own like bespoke. So

1:00 that's what I'm going to talk about. >> Beautiful. Well, I'll start sharing your screen now and then and yeah, the floor is yours. Thank you. >> Okay. Super. Well, thanks everyone for

1:10 joining. Um I will do a more thorough introduction in just a moment. I just uh kind of introduced what I'm going to talk about is yes there's you know MCP is stateless now with v2 which

1:21 simplified radically the the tasks protocol which you even saw a little bit of that just a moment ago with Alex but I come from temporal so durability is something that we think quite a bit

1:33 about and so I'll touch upon that because it's pretty prominent in MCP tasks v2 or in MCP tasks as a whole. Okay. So, um just a little bit more of a an intro. So, I have been doing this

1:48 this whole, you know, technology thing for a long time. I have lots of gray hair to show for it. I was very fortunate that I was at the right place at the right time when the whole

1:56 microservices movement was happening. I was at Pivotal. Um so, I worked on Cloud Foundry, helped bring that product to market, helped build that product really, I mean that product market. And

2:07 uh then I was fortunate enough to work on Kubernetes and I um spent so much time on it that Manning reached out and asked if I'd like to write book which I did. Um boy that was an experience and

2:18 now I'm a technologist technologist at Temporal which is all about distributed systems and that's really what we're talking about here all the time. So the agenda for the next uh 20 minutes or so

2:29 is I'm going to ground us in a very concrete example purchase order processing so that we don't just talk about things in the abstract that we can see things that are happening in real

2:39 life. I will give just a very brief overview of MCP tasks. I know you had that earlier, but in case anybody's joining late, just a quick overview so that you hear it from in my vocabulary

2:49 in the way that I talk about it super super quick, >> real fast on MCPtasks v1 and then I'll spend more time on MCP task v2. I'm going to do a live demo to show the V2

3:03 protocol, the client server protocol, and then we'll talk about kind of next steps actually because there's even more that I want to do that I don't think has been touched upon by Vicrim or Alex uh

3:15 in the prior two sessions. So here's the use case. Here's a purchase order processing, you know, workflow. This could be agentic [clears throat] for simplicity here. I am not worrying

3:25 about the agentic part of the interface or in agentic part of the client. I'm really just focusing on the protocol itself. And so creating this very fixed flow was a very easy way for us to focus

3:36 on that protocol. So after we have recorded that we got the things that we had ordered, um we're going to do a couple of things. We're going to update inventory and then we're also going to

3:48 pay an invoice. Um so I'm some kind of a merchant that resells products and so I've got an inventory system. And notice a couple of those things can happen in parallel. You'll see all that running in

3:59 parallel. And the important thing is that in this particular case, it's a very concrete example of one of these long running processes. So you can imagine that I might have an invoice

4:10 processor that is something that facilitates this very long process. It facilitates the submission of invoices, the waiting for approvals, the checking against policy systems, checking against

4:24 back-end systems and all of that stuff. That's a very longunning multi-step process that could be done as quickly if it's a very small item. It could be done as quickly as a second or two, but it

4:35 might take several days or weeks. And so it's a perfect example of what we want to do with tasks. And so if we wanted to put an MCP server over the top of that, previous to tasks where everything was

4:48 synchronous, we would have had to kind of break up that. we would have had to really show some of the innards of the MCP server, the invoice processor, and we would have to create very low-level

5:00 tools across all of those things. You'll see where I come back to the there's two different life cycles that we're going to talk about, and I'll explain that in just a little bit. But this is the

5:11 scenario that we're going to show. And I come from temporal. So the way that I would implement either the the client side, the PO process or the invoice process is I would use temporal.

5:23 And here's just a couple of snippets of how I would implement that server side. My invoice workflow, which is what you see graphically on the right hand side, would be implemented as a temporal

5:32 workflow. For those of you who don't know temporal all open source project, so I'm just talking about the open source here. and then doing things like being able to supply information like

5:43 both um Vicram and Alex talked about um elicitations. One of the mechanisms for satisfying elicitations into an MCP server um built on temporal is something that's called a a workflow signal. So

5:56 you can signal into it and you can also query status out of it which is important for things like polling. So you'll see where all those things come in. Um the key is that of course we're

6:06 talking about tasks and this is an asynchronous process. So we need to be able to support that. So real quick on the tasks protocol overview is at a high level and this is applicable to both v

6:18 V1 and V2. What happens is that in case somebody hasn't said it explicitly before there is not a special type of a tool called a task. It is that you have a tool that can be invoked either

6:31 synchronously or asynchronously. And there is a there's a kind of a dance that can happen between the client and the server to decide which way it happens. And Vicram actually showed some

6:40 of that in his demos. But of course, you're going to get back a task handle and then the client has that handle and they can interact with the longunning task via that handle

6:52 um including things like approvals. And so, yeah, that looks easy enough, but it turns out that when you get into um some of the details, it gets a little bit more comp complicated. And one of the

7:04 reasons that it gets more complicated is it's not just about sending back handles, but the specification for tasks intentionally explicitly says that tasks are durable, that they're durable state

7:17 machines. What that means is that once you start a task, it's expected to stay there and it's expected to know what state it's in. So while the protocol is stateless, tasks themselves are very

7:33 stateful because again they're long running and can have multiple steps. And so the task itself has to be durable and so there's some complexity in implementing that on the server side.

7:45 And I did a talk at the MCP summit earlier this year specifically on the server side of the tasks protocol and talked about how you can actually h what you can do how you can satisfy those

7:59 durability requirements and there's a QR code in a bit.ly if you want to see the video from that but what I want to do now is I want to talk more about the client side and more about that specific

8:11 client server protocol. not so much how we implement the servers but how we implement that communication mechanism between the client and server. So let's talk about V1 real quick and the parts

8:23 that there's parts that stay in V1. For example, this picture on the right hand side is that tasks have a life cycle associated with them. If you remember the picture that I showed you earlier,

8:34 the specifics around invoice processing, yes, that has a set of states that it goes through as well. It's been submitted, it's been approved, it is waiting on, you know, it's it it's now

8:45 going through and it's line items are getting paid, etc., etc. But tasks has a task life cycle and that life cycle includes working. So that means it's in process,

8:59 it's waiting for some kind of input. So it could be an input required or it's terminal. and terminal is either completed failed failed being one way of completing or completing in a successful

9:10 way or it can be cancelled. Now the way that this and then of course it had the protocol that between the client and server. Now at a high level this is what what Alex showed just a

9:21 moment ago. At a high level, it's relatively straightforward and we had these various APIs that needed to be supported by the server and the the server and the client. So you

9:34 could call a tool. Notice that is on tools, not on tasks. Back to my earlier point, you can get the status. You can figure out what the the status is. And that status, by the way, is the life

9:46 cycle. It's the task life cycle. So you'll get back a status of it's either working or it's input required. It's not going to give you necessarily the status of the the domain of the the task that

9:59 you're running. So task get is mostly about that status. you could cancel and then there was this list which in V1 these things were stateful and so you could go back and there was no

10:14 requirement that the client keep a hold of all of the handles but there was a requirement that the server did so part of the durability requirement fell on the server to even keep track of all the

10:25 tasks. Now there were some there were no APIs that allowed you to subset those filter those tasks those types of things. This is one of the I did some implementation with V1 and this was one

10:37 of the things that was quite painful and then it had task results which the task result by the way this is implied but just because you can do task list doesn't mean you should but as soon as

10:50 you task results was the channel over which the elicitation was flown was flowed was flowing and so this was extraordinarily complex and I did get this implemented I did get the client

11:02 side implemented but in doing so it answered the question of why didn't goose implement support for this earlier and it's because probably not a good idea probably pretty smart not to

11:14 implement V1 in a general setting so thrilled to hear that you're implementing V2 because it is as Alex said pretty solid this is just an overall picture of all that was involved

11:25 in the client side protocol the client side of the task protocol specifically around elicitation So you had this polling loop that was happening here. Then when you needed to handle

11:37 elicitation, you had this whole complex thing that needed to happen where you opened, you know, you did a task result which opened this SSE stream or it was open via, you know, standard IO and then

11:51 what happened if that went away, what happened if somebody responded and as Alex said, you could go back to a working state and then back into more input required. So, as a client, you had

12:02 to handle all of these things. Um, yeah. So, that wasn't any good. And then, of course, as you all know, we saw Angie's blog post in May. This was right around the time that I was finishing up my work

12:15 with V1, and I was like, "Oh, super excellent. I'm going to go over to V2." Nobody was supporting it yet. So, of course, Claude and I implemented the client client and server side protocols

12:25 for this. And I'm not going to talk about this slide because Alex just did a beautiful job talking about some of the other things that happened in V2. So let's talk about VA tasks V2. So on the

12:37 left hand side we had that in V1. On the right hand side we eliminated task list which is great and Vicram really pointed that out as well and he pointed out kind of the security implications of it. But

12:52 there are some other implications that are all around. I long history in distributed systems and scale out systems. There are also some implications on scale task list just

13:02 wasn't going to scale to thousands or tens of thousands or millions of tasks in flight. So that has gone away. Task update has been added and task result has changed significantly because it is

13:14 no longer the channel over which elicitation flows. What stayed is this task uh life cycle that we had from before. now because we can no longer have this

13:28 the server responsible for because the server is no longer responsible for keeping a handle. There's no get lists, get list or task lists. Um so the server is no longer responsible for holding on

13:43 to all of the in-flight tasks. There is language in the um specification in the extension that says clients should pro persist task ids to durable storage. So what this is saying is not only is there

13:58 a durability requirement on the server side but there's also a durability requirement on the client side. Now why this isn't a must I am not quite clear on. I still have a little bit more work

14:11 to do there and um am engaging with the team to understand whether that should be a must. But all of that means that this ugly client side protocol simplified into this. It's really quite

14:24 simple. You've got a polling. You've got something that polls when you're you need to handle an elicitation. you'd get back enough information from the server to wait, transfer that back to a user

14:36 request or go to another system to get the answer to that elicitation request and then you have an update API that allows you to update the state of the um MC MCP task and continue processing. So

14:51 with that, I have enough time to give you a demo. So I'm going to jump over here and do a couple of things. I am of course running this on temporal and like I said this is all open source. I'm

15:03 running the open source server locally here and I'm going to run this purchase order process and I'm going to submit and I only have time for one of these. So I'm going to go ahead and submit a

15:13 more complex example. It's the one where we're going to need elicitations a couple of times. So I'm going to submit a large invoice. And you can see here that it's popped up. Okay, here's my new

15:24 purchase order and it's running. And initially it was showing you the state the task state that it was in the life cycle state that it was in which was input uh which was working initially and

15:39 then it got to the point where it was waiting for input over here on the right hand side. I'm going to show you what's happening inside of the MCP task. So let me up refresh this and oh my filters are

15:52 not working. So I'll go ahead and go into the invoice. So this is the MCP this is the implementation of the MCP task. So this is the invoice processor. You might remember the first step was to

16:04 validate against the ERP and then based on certain conditions it was maybe going to ask wait for approval. We see that approval request reflected here in the UI. So we went into an input required

16:18 state. Let's take a look at what happened on the client side. I showed you a picture just a moment ago of what that client side protocol looks like. And here's the PO process.

16:29 So you might remember that the PO process was we're going to record the receipt of the goods. Then we're going to in parallel do three steps. Update the inventory, notify the requester, and

16:39 close the PO. And then we're also going to process the invoice as a part of that that I have carved off that client side protocol and put it into a separate process. And so going back up to that,

16:54 let me show you what's happening in the task tra track tracker. So this right here is a visual representation of exactly what's happening on the client side. So you can see here that we did a

17:04 task start, then we did then we started polling. And the first time I pulled um I didn't get any any response. It was just in a working state. So if I were to click on this when I pull, if I look

17:18 down at the results that came out of this, you can see that the status was working. So then I have it configured to pull every two seconds. And so it had a two-c

17:29 timer and then it pulled again. And this time when it pulled, it got back the fact that input was required. And it gave me all of the data that was necessary to satisfy. It essentially

17:42 gives me a schema for the data that is being waited on over on the MCP task side. So I got back that and now I'm in a wait state. So now my client says, okay, I'm just waiting for the input

17:57 required. Now we have some machinery that ties that task tracker back to the initial workflow. So let's go back into the UI and we'll go ahead and [clears throat] use the UI to approve

18:13 this process and let's see what happens here. The user decision of approval came in and then we submitted the result. So this is where we did the task update and now we're pulling. Now we're pulling

18:29 again. So we're pulling on this and again I have it configured. I don't have any exponential back offs or anything like that. I'm just pulling every two seconds. Well, what's happened in the

18:38 UI? Notice that over here, I got the approval for the invoice. Let me scroll that up a little bit. Got the approval for the invoice. Then I did the next step in invoice processing, which is to

18:49 reconcile against the ERP. And now I'm waiting on input again. And so if we come back to the UI, you'll see that we went back into an input required. This time I'm asking for a cost center. and I

19:02 submit that. And let me scroll up here so you can see what's going to happen. So I'm going to submit that which facilitates through the MCP client over to the server. It's submitted the

19:16 cost center and now it completed everything. Oh, and the failed here isn't that the MCP server failed. The failed here is this is probably I would call this a bug in the implementation.

19:27 It's that one of the line items failed to get paid. So what we're actually doing there, I'm just realizing is we're bleeding the semantics of the task life cycle with the semantics of the invoice

19:37 processing. Gonna be fixing that bug right away. Okay, so that's the main thing I wanted to show you. Let me just round out with a couple of points and then I will be

19:48 done. So just some advice, some very specific steps. So, if you're going to want to do MCP tasks, asynchronous MCP tools, you need to make sure that your servers are

20:01 durable. That's what it says in the spec. Make sure your clients hold on to handles because if the clients lose the handles, you've got an orphan task. There's no way to recover it. Now,

20:11 here's the big thing is that I mentioned that I'm really focused on scale and those types of things. come up with a technique for being able to hold on to lots of handles because

20:23 you might have millions. We can imagine some of those use cases and you of course are going to make those MCP calls. I mentioned this was a fixed flow, but it could be happening in an

20:36 agent. So you could be executing lots and lots of steps on the client side as well. And all of that needs to be efficient, which brings me to my final point. Here's just a schematic of that

20:48 previous diagram. These are the things that you need to implement. But here's the point. Going all the way back to this picture, I've said it several times. You're potentially going to have

20:58 many processes on the on the client side as well as many processes on the server side. I've spoken with customers who are literally running like millions, hundreds of thousands or even millions

21:11 of instances of agents that are doing let's say customer support and those types of things. And unfortunately, we tend to in the beginning especially think about things

21:24 in small numbers. But this gets me to my kind of closeout, which is what I'm working on right now is I'm working on implementing the notifications part of the MCP tests protocol. We don't even

21:37 talk about it that much. I don't think Vicram talked about it. I don't think Alex mentioned it, but I'm here to say that in order to scale this to the volumes that we think are going to exist

21:48 in some of these MCP use cases, we cannot have a million clients polling every two seconds or every 5 seconds or even every 10 seconds against millions of tasks in flight because in many cases

22:06 those tasks are going to be idle for long periods of time. So that's incredibly inefficient. You're using, you know, network bandwidth. You're using compute services and those types

22:16 of things. The important thing is while these things are idle, you want to consume no resources. That's the key. And the tasks part of the notifications part of the MCP uh tasks protocol shows

22:30 a lot of promise. As I'm starting to work on it, I'm finding some edge cases that need a little bit more attention that get us very close to even having to start talking about the different

22:41 protocols, the the different transport protocols. So, this is work that I'm doing. I'm I've I'm delighted to say that I got a talk accepted at MCPcon and Agent Con in October, and I'll be

22:52 talking specifically about notifications in that talk. All of this stuff that I've showed you here today is available in this repository. So, I invite you to take a look. You can run everything that

23:03 I just ran and can kick the tires and I would love to hear back from you. So, with that, yes, we are hiring and I don't think we have time for Q&A, but you know how to find me. I'm the only

23:15 Cornelia Davis at Temporal, so you can find me on LinkedIn and I look forward to chatting with some of you. Leo, I'm all done. I am not hearing you. I don't know if

23:28 it's just me. >> That was me. Thank you. Thank >> you very much for going into all of that. Hearing things from different perspectives is very helpful to be able

23:36 to hear tasks over again and then also with your example of the purchase part. We are a little bit over but we don't have next speaker. So I just did want to ask for when you're talking about

23:47 scaling that purchase part. I really like the whole uh let's go ahead and implement a push function rather than several pulls over and over again for any of that. Do you see anything on

23:58 security aspects though for handling that many purchase orders and things like that? Um I know the new SCP release has better authorization, but when you guys at Temporal are thinking about

24:11 scale, um are there any things that people that you'd like to hear back from people on on the security side? >> Yeah, I mean I think that's a great call out because what we do with

24:20 notifications in general is you have a single push channel. That's the thing. That's where you get the scale is you have a single push channel. Now that single push channel however doesn't mean

24:31 that a client that is interfacing with that single push channel is going to get visibility over everything. And so you definitely need to put some security around that single notification channel

24:43 so that clients are only getting the notifications on the things they have visibility to. The notifications protocol does have parts around expressing an interest in these are the

24:54 things that I want to get notification on. So that can play a little bit of a backbone. I'll be honest with you, I haven't I haven't gotten that far in my implementation to know concretely

25:03 whether that whether all of the tools are there or whether we need to augment with probably some of the security things that we've been layering on top of, you know, microservices and any

25:12 other systems for a while. >> Yeah. Yeah. Absolutely. I appreciate it. I also appreciate the call out for Agent Con and MCP Con in North America. Uh that will be happening on the 22nd and

25:24 23rd of October down in San Jose. If anyone wants to attend and speak to you in person and learn more in person, please come on out. >> Yes, super. Well, thanks. I'll let Leo

25:35 close it out. See you. >> Thank you very much for your time, Cornelia. >> Thank you. >> Great. As I last call out that Agent Con

25:44 coming up, we also have the worldwide for other ones coming up through as well. And then also there is a MCP uh certification uh if you go to uh the Linux through the Linux Foundation. So

25:57 that just released on the beta and that we're looking for more feedback on that and participants there. So please uh look into that and have a great day everyone.

26:08 >> [music] [music] >> Hey,