OpenAI DevDay - What Actually Matters for Builders

0:00 Okay, so OpenAI had their dev day yesterday and there are a number of sort of really interesting points in there. What I want to do in this video is go through what they announced and give you

0:11 my honest take on some of the different things that are there. I actually think there was some really interesting announcements, not only for OpenAI themselves, but also about how people

0:20 are thinking at the Frontier Model Labs themselves. All right, so first up is dots. So what actually is a dot? It's basically an always on agent that lives inside your chat GPT or at least chatgpt

0:35 and codeex for now. There was some really interesting things though where you can sort of text it, you can call it and stuff like that. They talk about that it has its own computer in the

0:45 cloud, its own browser. It can write and test code for you. It uses the same plugins that you've already connected to your account and all those sort of plugins and apps and connectors. There

0:58 are thousands of them there. One of the cool things here is you can add it to Slack. So in many ways you could kind of think of OpenAI basically covering off what anyone else has done in the

1:10 personal agent space. So I think the big thing here is that this is their first or at least their more substantial foray into personal agents. So the idea here of where it can basically do things for

1:26 you, it can be probably a lot more proactive than just reactive. Now, it's going to be interesting to see how that proactiveness actually plays out. Is this just a bunch of sort of cron jobs

1:38 going on underneath or is there something a bit more to it? Obviously, a lot of the ideas for this have come from things like Muse, like Grockbot, like Instinct, which is a startup that is

1:51 very hot at the moment in San Francisco. It's just raised about a billion dollars on a $10 billion valuation for their first round. And of course, the other two personal ones here would be Open

2:02 Claw and Hermes. And don't forget, while OpenAI didn't buy Open Claw directly, they did hire the creator of that. And you can imagine he's had a lot of input on where dots is actually going and how

2:15 that should actually work. So the interesting thing here is that this seems to both be a pitch that this is your personal agent but also perhaps more than your personal agent. They talk

2:24 about that you give it responsibility. So rather than just sort of go and do a task for me like book a flight or something, this is going to sit on top of things and manage and monitor things.

2:35 Whether that's like incoming bug reports, whether that's testing fixes, whether that's doing a whole bunch of different things. Now the interesting thing is that this is actually powered

2:45 by Astra. So it's powered by the top level model that they've got there. And it does seem in a very sort of open claw/h hermes kind of way that you're kind of expected to treat this as your

2:57 chief of staff. Now the usage model for this is really kind of interesting and worth understanding as well. OpenAI said that your dot is included in your plan. So talking to it doesn't consume your

3:10 token usage sort of stuff but anything that it kind of does on your behalf does. So, a dot operates on top of your chat GBT and your codeex threads, but the threads that it drives for you in

3:23 those particular apps or in its own computer, it looks like they're going to be threads that do count to your token usage. So, basically talking to it is free, but the work that it does for you

3:35 is not going to be free. I can still see that this is going to be a huge win for people who perhaps have never tried out Open Claw, never tried Hermes. And in many ways, Google tried to do this with

3:47 Gemini Spark after Google IO, but it seems to me that that hasn't taken off as much as they would have liked, perhaps in large degree because of the models that were actually running it and

3:59 what it actually had access to. You could imagine too that the actual dots or the dot that works for you is going to be a large part of the OpenAI hardware that supposedly they're

4:11 bringing out a product either late this year or next year sometime. And so if that product is like people have pictured where it's some kind of you know hockey puck or something like that

4:21 that you can then talk to that you can then interact with you could imagine that that is something then that people are going to basically put on their desk have it sitting there on all the time

4:31 being able to talk to their dot and if they feel the usage for that is free for the actual conversations but they're just getting build for anytime this thing goes off and does something for

4:40 you. I can see that being appealing to a lot of people who don't have a plus account, don't have a Teams account, don't have a Pro account currently. And like I've said in a number of videos, if

4:50 you're watching my videos, you're definitely one of the earlier adopters. I think some of the real huge growth that we're going to see over the next couple of years with these particular

4:59 companies and with AI itself is just the normies or the sort of masses are going to come along and start using these things in their day-to-day life. All right, so let's look at some of the

5:09 developer stuff that they launched. So that brings us to sort of agents API and this is basically their managed agent runtime. So just like we've got claude managed agents, we've got a number of

5:20 companies that are doing this and I've made videos about these specific things. You basically get a harness hosting memory sort of multi- aent controls and now they've added computer use to that

5:32 as well. I do wonder how much is this a meto product basically where they just don't want to lose market share to Anthropic who's been offering their claude managed agents since much earlier

5:44 this year and it's interesting that this does seem to be the same stack that powers codeex and dots just like the clawude managed agents is using a lot of the same stack that powers claude code

5:56 and co-work so the way I think about this one is that there are a lot of companies out there that are using the open AI models whether Whether that's directly, whether that's through AWS or

6:05 Microsoft, this gives them a way to build vertical harnesses so that an agent for insurance claims, an agent for sort of contract reviews, all of those things are going to be possible. And

6:17 it's basically going to be running on a runtime that's kind of been rubber stamped by OpenAI as to what they're saying is the best way for running their own agents. The second big developer

6:29 news here was the whole decisions API. And I kind of feel this one really wasn't a surprise. I've made comments before that probably most of the Frontier Labs have Jev models in

6:39 development after seeing just the reactions to the whole idea of a decision model. And yet again, like I've said in a couple of my videos, it does seem that the people are settling on the

6:50 name of decision models with OpenAI actually calling their one the decisions API. The idea here is that this is a Luna model. So again, you don't need a huge model to do this. This is something

7:01 that we've looked at in my videos with the open models, etc. Certainly, Luna is actually a bigger size than a lot of the open models that I've shown. And pretty much the idea here is very similar to

7:11 Jev. You've got a model, you give it a predefined set of options, and it basically sort of picks one out. Interestingly, they are launching with image support, and I kind of feel this

7:21 is a no-brainer here. You don't need a big model to do it, and you don't need a huge amount of training. OpenI didn't really even hide that. they sit on stage, it's the Luna model. And I think

7:31 for a lot of organizations that aren't going to want to add another new company, aren't going to want to run something like this on prem, this just becomes a no-brainer for them to

7:40 basically use the OpenAI version of this. Now, another big developer announcement that they had there was the new model. Now, almost every year we get a new model and stuff like that. And at

7:52 first, my gut reaction was like, okay, GPT 6.1 Soul, remember Astra is the really big one. Solers, their version of sort of Opus. My first reaction was like, okay, so what? Does this really

8:03 kind of matter here? But it was interesting that just like Opus 5.5 seems to have improved massively on Fable, they're saying that GPT 6.1 Soul is actually getting up there on par with

8:16 the Astra level here. And the other thing here is that this is like a fifth of the actual Astra costs. So it's $2 per million input tokens, $10 per million output tokens, and then

8:29 amazingly only 10 for cing that input. So when you compare that to Astra, which is $10 in, $50 out, this is an 80% saving for a model that's supposed to be on par with that Astra model. Now, it

8:44 does seem that just like with Fable, there are certain kinds of things where if you're maxing out the thinking on Astra or on Fable, the second tier of models is going to be just slightly not

8:56 as good there. The other thing that I thought was fascinating here was the actual price for caching. So, it's a 95% discount. And we know that sort of agents reread the same context over and

9:07 over again, the system prompt, files, conversation so far. So for agent workloads, the cash price is going to be the real price in here. And on a personal note, I noticed this yesterday

9:18 with something that I was working on where I' done 250 million tokens of using the Opus 5.5 model. Yet about 96% of all the tokens were actually already cached. So the price wasn't anywhere

9:32 near what you would think it would be if you're talking about 250 million tokens there. So out of this, I think if you're running agents on OpenAI, Soul 6.1 is probably going to become your new

9:43 default model there. All right, next up, that brings us to codeex. And there are a few things that people have been asking here for. So codecs on Linux, multiple folders in one project, codecs

9:53 on your phone. Samman basically announced those very quickly, even before they got to probably some of the interesting things in here. Big one for me here is that whole idea of codecs in

10:03 the cloud. So this definitely enables you to basically start a task on your phone, pick it up in your browser, run with it over a lot of different surfaces, and it was kind of funny in

10:14 Roma's demo for that being literally like, okay, rewrite the entire backend in Rust and then just sort of leaving it and walking away. Interestingly, they've also brought voice into the app and to

10:26 the CLI. So this is their birectional audio powered by GPT live. So it's not just doing sort of dictation. Unfortunately, they had some hiccups with that on stage. But my guess is if

10:37 you're like myself or most people who are using these agents nowadays, you're talking to them and not typing to them most of the time. Anyway, there also some nice things that they showed around

10:47 computer use and allowing codecs to actually use that, whether that's for sort of getting app shots as it's actually making something for you and some nice stuff around security of where

10:57 it can basically scan for vulnerabilities and prepare fixes for you to review as it's actually going through your codebase. All right, so next up was one that I think is really

11:07 kind of interesting and this was ultraast. So this is Astra at eight times the speed of the normal API for six times the price. Now they're claiming that this can do around 300

11:18 tokens per second. I'm guessing this is basically the running on Cerebrus version of these models. And as we're all starting to see with running these agents, these agents can take hours. So

11:31 I guess if you've got the budget to pay for this, being able to get something done at a really fast pace may be worth it. I'm not totally convinced though how much people will actually pay for the

11:42 the six times the price API. They mentioned that this is out on Astra and coming to Soul very soon. I guess if you were to take the Soul 6.1 at six times the price, it's only slightly more

11:55 expensive than the normal speed Astra was before. So like that maybe it is a good tier. Let me know in the comments. I can certainly see that maybe there are people that are quite happy to pay that

12:05 premium to get the tokens a lot faster etc. And it is probably going to make sense where latency is the product. So for anything that you're doing sort of live editing or interactive demos or

12:18 where there's actually a user sitting there waiting for something to happen then those tokens being that much faster probably is going to be worth it. And this kind of ties us into one of the

12:28 things that I think is probably controversial that they announced that there's a new Pro 500 plan. And as I was watching this, I only kind of twigged what it was as they were sort of rolling

12:39 out that there were pro 200, Pro 100 plans as well. So this is a 500 being $500 per month here. And this is going to have 25 times the usage of Plus. It's also going to have access to ultraast

12:55 inacht and codecs and interestingly that usage is going to carry over into partner apps. So that's something that's going to be interesting to see. Now they reopen the pro200 plan. So the $200 plan

13:08 they actually closed that after Astra for a while. I think they just didn't want people just hammering it and stuff like that. And the sad thing here is and the thing that perhaps is going to anger

13:17 a lot of people is that this has gone from being 20 times plus down to 10 times plus here. Now we know that usage limits are something that kind of fluctuates anyway. So, it's going to be

13:29 interesting to see how this actually plays out and does that drive a lot of people who were paying for the $200 to move to the $500 plan or does it have the opposite effect and drive more

13:42 people to go over to Anthropics plan? It does seem at this point that the pricing is kind of steering you and that really now if you're going to go for the sort of top tier models, you're going to have

13:54 to be paying quite a lot of money to do that. All right. So, the third part of the keynote was about helping you to find customers. And here basically Samman was pretty candid that they've

14:04 tried different versions of these kind of things before with mixed results. And just to recap those, we've seen a bunch of these things. They first started out with ChatGpt plugins back in March 23.

14:16 They had GPTs for a while from sort of November 23. They had the GPT store. They then went and sort of switched off all the original plugins. Around this time last year, they announced apps in

14:29 chat GBT and the whole sort of apps SDK. And I don't know about for you, but I haven't really seen apps take off here at all. So, it does seem to me that this is something that's still definitely a

14:40 work in progress. And I think Sam Orman's statement of saying that they'd had mixed results with what they've launched is in many ways an understatement here. So, what have they

14:50 got news? So I would say the big and most interesting piece here is the whole sign in with chat GPT where users can sign into your app and use the tokens that come with their chat GPT plan on

15:03 your particular app. And I think the argument here is that that allows you to basically ship sort of like a free core app that because you're no longer having to pay for the inference of tokens. you

15:16 can sort of give people a taste and then you can upsell to other parts of it without it costing you a fortune. So, I think this really opens up the door to a bunch of new premium products that we'll

15:27 see that use this kind of thing. The challenge is going to be though, do we very soon see a signin with Claude and then are sites going to basically have to allow people to sign in with Claude

15:39 or to sign in with Chat GPT or to sign in with Gemini? All these ideas are trying to do what the Facebook login button did sort of back 15 plus years ago, but we don't just have one major

15:50 player here. We've actually got a few different players. It's also not clear what are you actually offering if someone comes to your site and they've only got a free chat GPT plugin. uh you

16:00 then sort of stopping their inference halfway through a question or halfway through some particular task and having to tell them hey you should go and pay for chat GBT as opposed to you should

16:12 pay for my product on my site. All right next up a plug-in extension. So these are basically full apps that run inside chatbt and codeex Figma and Adobe both showing versions of their products

16:24 running in there and supposedly chatbt is going to suggest your plug-in in the middle of a conversation. I think we're still in the same place as last year where we don't know how it's going to

16:34 suggest that I use Adobe Photoshop over some other hipper cooler image editor that happens to be in their directory as well. Is that going to come down to whichever developer pays OpenAI the

16:46 most? I don't know. I guess we'll see with time. They also launched an OpenAI marketplace with a bunch of different partners including people like Code Rabbit, Notion, etc. And I thought it

16:56 was kind of funny that Sam Alman actually made the point that because B 10, who is an inference provider, is one of these partners in the marketplace, you will be able to access opensource

17:07 models in the marketplace. My bet is that's something that you're not going to see on Claude anytime soon in any way, shape, or form. So overall, I would say that the event was actually very

17:17 interesting. There were lots of things showing sort of ideas of where things are going. I do think that things like the agents API is going to be something that a lot of people are going to use in

17:27 organizations to make their hosted agents if they're already an OpenAI customer. I do think it's going to be really interesting to see how do plays out and three months from now are 10

17:38 million people using it every day or is it something that people have just basically added onto the site and it could be that dots becomes a new part of co-pilot or of some kind of enterprise

17:48 apps which is something that they actually talked about making sort of work versions of these kind of helpers as well. Anyway, let me know in the comments what you thought was

17:56 interesting there. I know a lot of people are going to bag on OpenAI and criticize everything that they did. And if you've listened to the end here, thank you very much because I know this

18:05 video is probably not going to do very well. But I do feel it is very important to see what they're thinking about how things are going to play out. We know that the all you can eat buffet of

18:15 tokens cannot go on forever as costs go up. We know that lots of people are trying to find the killer app for AI so that you can charge a premium and not just focus on selling tokens all the

18:27 time. And I think OpenAI's dev day is always interesting to get a sort of state of play of where things actually are both for consumers but for us as developers as well. So anyway, as

18:40 always, if you like the video, please click like and subscribe and I will talk to you in the next video. Bye for now.