JSON Schema 2020-12 and the Contract for Context | ​Ola Hungerford | MCP Release Party - Seattle

0:00 There we go. Thank you. Um, yeah. So, my name is Ola and um I'm a principal engineer at Nordstrom and I'm also a maintainer uh for MCP.

0:14 Um, I've been working on it I want to say like around a year and a half now, maybe a little less than that. um kind of got started working on NCP inspector just wanting to fix it and then things

0:26 escalated and you know got more and more involved over time and I think the the main changes I want to talk about today like it was actually hard to settle on like what aspects of this I wanted to

0:36 talk about because I think it it's not getting as much air time as like the obviously huge things like stateless and uh you know taking sessions out of MCP and a lot of the other changes but I

0:46 think it's going to have a big impact on uh tool line. Um, so yeah, my my slides are not uh Age of Empires themed. They're just here to remind me what to say. Um, so kind of

0:59 before uh the current spec, um, for a while I was noticing that there's a lot of issues related to uh trying to use one of the most popular MCP SDKs, which is a fast MCP. Um, and sometimes those

1:14 would actually pop up like in the inspector bugs and be like, why isn't my server working? And it would turn out it's because actually it isn't um it's doing things the spec doesn't support.

1:23 And it would turn out that like over time, you know, we're seeing that uh Pyantic would, you know, translate uh schema outputs and uh all these different like schemas trying people

1:34 were trying to use with tools um into a subset that wasn't uh actually supported. Um, and so like as time went on, you know, like we actually created like all these rappers and everything to

1:45 make sure that um, you know, we could actually speak to different clients and that servers could actually uh translate some uh their tools into something that was actually compatible with the

1:57 protocol. Um so like from an output schema perspective like uh one of the things that we were forced to do in output schemas is uh wrap them in an object and

2:12 use a limited subset. So, you know, if you had a weather tool, which is not the most exciting tool, but you know, is kind of an example of something where you might return some data in an array.

2:22 Um, you had to, you know, wrap that in some other object. Um, which, you know, that's kind of awkward. Um, but really there was a lot of, uh, limitations, right? There was, uh, very

2:34 few allowed keywords on the input object for tool inputs. Um, and there was just a lot of different uh keywords available in uh JSON schema uh 2020 that weren't supported in MCP.

2:51 Um, so I don't know if it's like SE or SCP. I'll just say SE. it. Uh so SEP 21106 was a uh proposal that I didn't author but um I sponsored it because I was noticing all these kind of ongoing

3:07 issues in the ecosystem and really thought that it made sense to kind of actually allow uh a standard the JSON schema dialect that already existed to be more aligned with uh what you could

3:20 use in MCP. Um and so what actually changed um as of today uh as a result of this uh this up is uh in input schemas like you still have to use an object um but you

3:33 actually gained everything else in the JSON schema 2020 2012 uh dialect. So there's a lot of different like conditional keywords um things that you can do to be more expressive and also

3:43 you can uh use references and other things that you couldn't use before um to kind of be more efficient about how you define your schemas. Um, and then an output schema, which this is the part

3:53 that was hard because uh this made me want to make a whole presentation about uh structured content versus content in uh tool responses, but um we didn't really have time for that. So, um you

4:06 just want to know that um you don't have to have the same restrictions around what you return in the output schema. Um, I just wanted to mention a little bit that I think

4:20 it can take a lot of kind of patience and waiting for right time, right place with a lot of these proposals. So, there's things that can make a lot of sense, but they just take a lot of time

4:31 to kind of, you know, for the right for it to be the right time for something to be actually like accepted or even like, you know, reviewed. And so um in this situation uh was definitely an example

4:44 of that because um John Mc McBride who is the person who originally proposed this uh created a PR in June 2025 and uh we didn't even have SEPs then like it wasn't really the governance model that

4:58 we have today. um it went from an issue went to a PR uh got eventually got converted to the SE format and finally landed as the the SE that got accepted. Okay, so I was trying to illustrate kind

5:15 of like what is the problem we're actually trying to solve with some of these uh keywords that we can use now. So if we're thinking about like in the GitHub MCP server, uh there's a tool

5:26 called project read and uh in the description, you know, it's an all caps request, but it's not really a schema constraint. It's like it's saying please critical,

5:36 you know, here's what uh always provide, you know, this if this is true, uh this is mutually exclusive. Um you know, it's kind of explaining those constraints in in the tool description.

5:50 And then another example of a tool that's kind of has to add a lot of um you know extra validation kind of more stuff in pros to actually try to get it to populate the right parameters is uh

6:01 this actions run trigger tool. And so um you know you have all these real constraints that are very conditional on like what the inputs should be like depending on different

6:14 conditions. And so if you break it down kind of to what you're trying to do here um you can kind of see like okay what method am I trying to use uh what parameters are actually required

6:26 um and then you might define it using the all of keyword to say like okay these are the different conditions that need to be met um and so the model can see that and it's like okay here's the

6:37 actual conditions designed as a schema and then you know assuming it supports this it can actually kind of understand uh more structurally how it's supposed to work.

6:51 And actually the part I want to emphasize isn't even um kind of like does can the model understand it. It's also can you just validate it without even waiting to uh have the LLM, you

7:02 know, try to use the tool. Uh because I think there's a lot of things where if you can just use validation um you actually don't need to make that round trip. you don't need to like spend

7:11 tokens uh and prayers trying to understand like okay is this actually is this tool going to populate things the way I want. It's more like you can just kind of validate um is it actually even

7:20 creating the schema correctly um and kind of go from there. And uh the other bonus thing that I was going to go into but might you know on different time is that um if you have

7:33 something structured with using an input schema or if you have uh structured content using uh output schema um you know input schema can become a callable signature and you can actually convert

7:45 that to executable code more directly. So if you want to use like code mode and all those different methods of using MCP it makes it a lot more compatible. Uh so the part I got really excited to

8:00 dig into uh because this is all great. It's like oh it it makes so much sense. It's going to be standardized and then you're like well does this actually work? Because I mean I think one of the

8:09 reasons why um originally these keywords were so limited and tool definitions is that um you know LLM providers, different hosts, different endpoints might not even support them. Um, so I

8:22 did kind of a bunch of different trials on a few different models, few different host providers, um, or a few different providers, few different hosts, uh, a lot of different combinations. And this

8:31 is just kind of a graph of, uh, how well each of these keywords were actually supported. Uh, which was actually a pleasant surprise because I mean, yes, there's some red dots or

8:41 orange dots, um, but generally like there's, you know, pretty good support from the um, at least in the sample. Um, and I have a link to uh kind of a full findings and data set

8:54 that I'll share out in the presentation because it's really interesting to see like for all these different uh models like is it really the model or is it um how the provider is actually you know um

9:05 allowing you to use the model. Um, so I won't go super deep actually into these examples because I'm not sure if they're like super illustrative presented this way, but like I thought

9:18 it was interesting to look at like GLM 5.2 and say like, okay, well, let's test it on a bunch of different providers and see like how well it actually supports populating these tool inputs. Um, and

9:29 one really surprising thing is like the first party endpoint actually was one of the ones that didn't support um or like actually uh refused the schema and were produced invalid arguments. So um it

9:40 really depends on kind of how that uh host has actually decided to support these inputs. Um, I also wanted to mention like I think I'm not actually sure of like the

9:54 split of like you know how many applications might be using let's say like one type of API for a provider versus another like for example an open API or open AI um you know like there's

10:07 a responses endpoint and then there's a completions endpoint and um there's a bunch of these keywords that were uh refused on like the completions endpoint which is like the the legacy endpoint

10:18 and this was like independent of the model like this was just what the endpoint or what that um what that supported or what that didn't support. Um, but then like if you switch to

10:30 responses, all of a sudden, you know, the one of keyword would work. And I think another thing I want to call out is uh part of what's going on behind the scenes because I forget this

10:43 sometimes is like when you have uh kind of this nice schema, it actually gets um rerendered uh you know depending on how you actually uh implement kind of the chat template and do the tokenization.

10:57 Like it can end up looking very different from the original uh JSON schema. And this uh helps explain why there's so much variance between like who or what

11:10 is actually serving the model. Um I won't actually read through all this. I just want to say a disclaimer because when I link to the uh kind of the experiments I did, I don't really

11:24 want that to be seen as like, oh, this is a ranking of all the best providers and all the best models because it's just kind of an illustrative snapshot. like there's so many different ways that

11:32 you can, you know, call a given model, especially with all the openweight models. It's like, you know, and all all this stuff is constantly changing. So, I think, um, one of the things I want to

11:41 emphasize here, um, besides just kind of the standard, here's how you translate your tools from one format to the other. here's how you uh you know adopt the new protocol as a a server author or a

11:53 client you know builder is I think you kind of need to explore uh you know how things actually work from like a data perspective. So, if you can um look at the specific SDK you're

12:05 using, uh if there are specific tools that aren't uh doing what you want, you probably want to dig a little deeper and try to uh see kind of where it's actually failing.

12:20 Um so, I think there's a lot more coming soon about this topic. Uh I have some more interactive stuff coming out with like uh what SDKs actually do. um especially as it relates to like uh

12:31 structured content and um you know how these different inputs uh uh actually get used in real examples as well as kind of an article going deeper into a lot of these topics. Um I was trying to

12:42 think of like where to follow me but I think LinkedIn is one good way uh also server on GitHub um and you can find me on the MCP contributor Discord server which is sometimes hard to find. So, if

12:55 you go to the actual official uh documentation, there's a community section that has a link to that. Oh yeah, and shout out and actually Bob

13:09 is here who worked on this a lot more than I did. Um as of today like there's a new uh MCP inspector which supports both uh legacy modern protocol eras like I think it has actually full support for

13:25 uh the latest protocol which um was a little harder to do before but now we actually support like all the different features I think so far. Um, so yeah, thanks Bob and Cliff who isn't here, but

13:37 I I wanted to call it the Cliff Spectre. Maybe the Bob Cliff Spectre because they like basically built the whole thing. So yeah. Yeah. And that's all I have.

13:56 Any questions? Okay, that's great. Now we will >> There's one one question. >> Yeah. Okay. >> Uh I'll just hand over the mic.

14:15 >> Um yeah, hey Ola. Um so it's not really a question, but uh I was actually going to ask Den if there were improvements to the input and output schema. So that's that's awesome that you covered that. um

14:26 just as like in general as well I was curious if if in the newer version of MCP if there's um any additions for um standardizing with hotel so I work a lot in the observability community and

14:38 that's sort of a big thing um with MCP servers we've been like adding a lot of different spans for um just tool calls in general and then also outputs from servers and agents so like security

14:50 outcomes classifier like REI classifier outcomes so I wanted to I was curious I guess for anyone that like presented if I haven't dug into it but if in the newer version if there are more

15:00 instrumentation uh specifically in the built-in uh yeah so there's uh there's a lot of interest in this area and actually I think there's some uh kind of proposals

15:14 uh that are open in the repo and I'm expecting I don't think there's like a working group around it yet but I'm pretty sure there will be it's just kind of the current spec probably

15:24 uh kind of you know push every a lot of things aside as far as like things that weren't involved in the spec itself. But um but I'd expect that like uh if if there's a new working group soon for

15:34 that that that's going to be something that's at least it generates a lot of um feedback and kind of like examples of like what what do we want to uh explore there. Uh but right now I think I think

15:46 it's more kind of at that stage. >> Sounds good. And then you referenced SE 21106. Do you have links to that or some I would love to kind of dig into that more. That was related to the input and

15:55 output schema standardization, right? >> Thank you.