I Made Codex and Claude Code Build the Same App. One Clearly Won.

0:00 So, I had Cloud Code and Codeex build me the exact same app. I gave them the exact same prompt, but the results are extremely different. Not only the outputs, but the way that they actually

0:08 got to the output, very different. One of them took 3 days, one of them took 5 hours, one of them spent $3,000, one of them spent $800. So, today I'm going to break down the two outputs, why they're

0:17 so different, and it's very clear to me now what Codeex is better at, and what Cloud Code is better at. So, by the end of this video, hopefully you have some clarity. Let's not waste any time and

0:24 get straight into the video. All right, so real quick before we start going over the outputs, let me just show you guys the actual prompt. I utilized a slashgoal and I did the slashgoal inside

0:32 both codeex and cloud code. Gave them the exact same prompt. So here it is. I'm not going to read the whole thing, but here it is. I basically said build a productionready originally branded type

0:41 form alternative. So we're kind of trying to clone type form here. I said orchestrate specialized agents throughout three phases. The research phase, the build phase, the verify

0:49 phase. Now looking back, I'm not saying this is the most optimal prompt. I honestly, if I was to redo this, I would probably add another phase between research and build that would be all

0:59 about planning and mapping out the whole flow. I honestly think that that would result in a much better output from both of these systems. But we'll take a look at them in a sec. Anyways, I'm not going

1:08 to read this whole thing out. You can screenshot it. You can reuse it if you want. I do want to call out what I put at the end though here, which was do not stop at a prototype or first successful

1:16 build. Continue researching, building, testing, breaking, fixing, and retesting until the app is genuinely complete. So, I basically just wanted something that was like production ready. Maybe we

1:24 could go to market the next day. Cool. So, like I said, I gave both coding agents, Codeex and Cloud Code, the exact same prompt. Okay, so let's take a look. I have not looked at either of these

1:34 yet. This first one I had to do on my Mac because I was actually out of town. I was traveling when I kicked off this first one and then I had the idea to like, oh, you know, I should have the

1:41 other one do the same thing. So, anyways, this is the first version that we have. I haven't actually tested any of this yet. I haven't clicked through. So, we're getting the real raw reaction.

1:49 My first reaction here is that this is called real form and it looks pretty solid. Like honestly, I because I've built a lot of sites with AI, I can tell this was AI generated. You know, we've

1:58 got the hero image over here, which is still pretty a pretty cool UI element. We've got the text over here. We've got this little pill. It just feels to me like AI, but the background, the depth,

2:08 it's not too bad, right? Like an average person probably wouldn't look at this and just assume, oh, AI vibecoded site. So, we've got experience, which will take us down a little bit. We've got

2:17 reliability, which takes us down again. We have pricing, which goes down to the bottom. We can build a private draft, which will take us to a signup page. I'm going to blur this out right here

2:26 because it says it basically gives away which agent built this version. And we can also click right here to start building. So, let me just sign up real quick for an account and then I'll show

2:36 you what the experience inside looks like. And by the way, this whole signup page, not too bad at all. I like this vibe. Okay, so we are in a demo workspace because it didn't have any

2:44 keys to make like real authentication and real whatever. So this is a demo workspace. Let's take a look at how this thing runs. So we can make a product launch survey. We have the ability to

2:55 structure how this works. This is the welcome screen of our form. So let's play a launch. People remember we can play with this text right here. We can put variables. So we can put a score or

3:05 a price. Interesting. Okay. So, this is how we like insert different elements and we can play with the sizings on them. Okay, very interesting. I don't know how I don't know how we remove

3:15 these. So, like if I wanted to remove these elements right here, these two boxes. I'm not exactly sure how I do that. Do I click delete there? Delete. Oh, this would delete the entire thing.

3:24 No, I don't want to do that. We can add text right here. Okay. So, I think this is a little bit honestly like looking at this, it's a little bit overwhelming. Like I'm kind of confused. There's a lot

3:33 going on in this UI. I can change the label of the button. Cool. We can upload an image for the welcome screen. Okay. So, I put in an image, but it actually says the image preview is unavailable.

3:45 So, this just goes to show that even though there was testing, there's so many different scenarios that agents might not think to actually test, especially if you're not the one driving

3:54 those tests. So, that's kind of like a bug that we already found, right? You can put alternative text. You can change the focus and the placement of everything. Okay, so that's the home

4:01 screen. Let's see what's on page one. So, this first thing is, what would you most like to improve? And once again, this is just too much going on. I mean, clearly this is a multiple choice type

4:11 of answer. As you can see, we could have made this also short text. And yep, let's change that to short text. There's still all of these variables up here, which I don't love. I mean, this is

4:19 supposed to be the title. I just think that there's too much going on, right? Like, this is a description. Please answer in one sentence max. People can put in their answer right there. But

4:27 we've got short text, long text, email, phone number, website, contact info, opinion scale. Okay, let's see what that one looks like. It's asking me to confirm. And I don't know if you guys

4:35 realized, but like when this pops up to confirm, it kind of like pops up over here on this left side, which also kind of feels like a UI bug. That's not perfect. We can do a matrix. We can do a

4:46 file upload. So, there's a lot of things that it is letting us do. So, I think that this agent did really good about thinking about kind of like from the admin side the data you want and maybe

4:56 the way you want the data to be collected, but from a UI perspective, like from the user perspective, making forms, this is confusing. Like I think this would probably have me churning out

5:05 really quickly because of how simple other things truly are like type form. We can have multiple pages. Okay, so I think we're starting to get the gist here. We can set up a workflow, right?

5:13 So, we can make kind of like logic based conditions within the form, which obviously is very important. We can set up a theme. So corner radius. We can do different colors. Which honestly though,

5:23 if I look at this, like this isn't letting me click. So that's really interesting, right? Like this doesn't really seem to be even changing anything. So that's not good at

5:35 all. Accessibility, we can see if there's anything blocking publishing. We can connect this to a web hook, email notification, partial response. We can share the actual link to the form. You

5:46 know, I think because it assumed that this is basically just a demo, it's showing me the UI, but it didn't actually maybe build out like the full functionality of this. So, I don't love

5:55 that. Right now, I do have to say that I do think that there's promise here, but because it built that out purely with the idea that it's a demo, even though I told it, hey, I'm looking for this to be

6:04 like a a real genuinely complete product. That's not great. Okay, so now we're hopping over to my PC where we're going to test out this other version. So, this version's called Fora.

6:13 immediate reaction is that this was not really being thought about from like a design perspective. I don't even know if this is supposed to be I mean this looks like it's supposed to be a landing page,

6:21 but this thing is just brutal. It's brutally ugly, right? Anyways, build beautiful one question at a time forms. Share a link and watch the answers roll in. Self-hosted, private, and fast. So,

6:30 let's go ahead and get started for free. I'll go ahead and create an account. My name will be Bob. Oh. Oh, nope. My email will be bobtest.com. And my password will be 1 2 3 4 5 6 7 8.

6:41 And that will be that because it's at least eight characters. Sign up. Okay, cool. So this, yes, I agree this looks vibecoded, but at least as a user, I know exactly what to do. I'm not staring

6:53 at something and immediately overwhelmed. I've got my account down here, which okay, that is another UI bug, right? Like it's not letting me I don't know

7:04 what this button is for. Okay, it opens settings, but I'm not able to I don't see log out. I assume that logout would be down here, but we can't actually access it even if I change the sizing.

7:15 So, that's one bug that didn't get caught, unfortunately. We have different workspaces. So, right now, I'm in a workspace up here, but I can create another one. So, let's call this one

7:24 business. We can create another workspace, and we can switch between those. Okay, that's pretty cool. That's pretty slick. Let's go ahead and create a new form. Okay, this is very nice.

7:33 This is way less intimidating. I actually feel like I know what's going on. This looks way more like a type form. Let's see. Default display mode is either conversational or stacked. So I'm

7:42 assuming we can either have it be one at a time or we can have all questions showing in one form. So we can also have a progress bar showing up. We can have question numbers showing up and this is

7:52 responding. We can have keyboard hints. Um we can have autosave and we can have capture partial responses. Okay, cool. And over here we're able to just customize the stuff easily. So page one,

8:02 first question goes here. What is your mood? Okay, cool. that pops up. Description happy. Okay, this is called question one. We can make it required. We can

8:14 have max characters pattern blah blah blah. Um, where do we control the field type is? Oh, right up here. Short text. Okay, this is showing us. Oh, I think it's because it's like a default. So, if

8:23 we add another one, there we go. So, when we add a new type of content, that's where we control what it is. Email, phone number, website, drop down, picture choice, net promoter score,

8:33 opinion scale, rating, ranking. Okay, so these are pretty cool. Let's see if we go ahead and do like an opinion scale what that looks like. Um, okay. So, it made it, but it kept the

8:44 same questions already. So, Oh, okay. No, it didn't. I just had to switch over to it. Okay. So, this is saying, "How much do you agree?" Now, one thing I just noticed though is this is keeping

8:54 the number here as one. So, like this should obviously be two. Um, how much do you agree though? That looks good. Opinion scale. Okay, let's try to add something else inside a website. Um,

9:05 right here. This is a bit of a bug. I've clearly selected this one, which is what is your website, but it's not showing that. I have to click off of it and I have to click back on it. So, that's

9:12 another bug, man. Like, you really cannot underestimate how much testing has to go into finding bugs. There are so many different bugs and it's not as simple as just telling cloud code, hey /

9:22 goal, build me an app. But anyways, this is giving a keyboard hint. That's pretty cool. Once again, I'm a little confused why it's not fixing these page numbers or question numbers. It's showing that

9:31 all of them are just question one. But what I've noticed is that like up here it says one of three and then if I go here it says one of two and up here it says one of one. So there's some other

9:42 logic bugs going on. But anyways this is honestly a much better experience. Let me go ahead and do let's see what else we got. We've got logic. So once again if they answer a certain thing we can

9:52 route them to a different place. We can add these branches. We have design so we can change the way the form looks. Okay that's pretty cool. You can also upload your own themes which is pretty cool.

10:01 Let's just say we like this dark theme for now. Now, one thing I don't like about this design is once you click design, you can't easily navigate back. You'd have to like, you know, hit the

10:10 back arrow. But once you click design, it puts you in this new UI where you there's not like a button to just navigate back. So, that's that. We can share it. We can look at our results.

10:18 Okay. Same thing. I guess we can click up here to navigate. We can go to web hooks. Same thing. It has this issue where once you navigate into one of these versions or one of these buttons,

10:28 you can't nav back very easily. So, that's a little bug we need to fix. We can change the theme from up here which is pretty cool. But anyways, let's go ahead and publish this. So when we

10:36 publish this, it gives us a link. And let me just call this form real quick. Test one. So when I publish this, um, I'm able to then open up a new page. And here's the form. Okay, this looks like

10:49 type form, right? So I am happy. I agree. Three. My website is um Google.com. And that is our response. Let's go ahead and see if we go to results. Oh, nice.

11:03 We got an actual result. I guess we're just seeing Oh, because this is just the insights of drops, right? This is if I go to summary now, I can see the actual answers. If I go to responses, I can see

11:12 each individual submission and where it came from as well, which is pretty interesting. But that's not bad. And if I go back to my dashboard, we can see our forms. Um, I think that I I thought

11:21 I titled this one. I guess that didn't save. So, I'll just call it one. Oh, I have to publish it. Okay, republish it. Dashboard, it's called one. If I go to a different workspace that doesn't exist.

11:31 Okay, cool. So, like this obviously is far far better than the first one. The first one from a design perspective I think was better, but as far as functionality and doing what I wanted it

11:40 to do, this one definitely takes the cake. So, now I'm curious, what do you guys think? Which one do you think was made by which AI? Well, let's just start taking a look at the results here. Okay,

11:52 so we've got Cloud Code, we've got Codeex. There's a few things that we're going to go over. First of all, let's do the reveal. Which one did Claude Code make? Claude Code made Fora and Codeex

12:02 made real form. So Claude Code made this one. Claude Code made this one which from a design perspective wasn't as good. But from a you know design of the actual like method, the functionality,

12:15 the actual output definitely much better. And I will be honest with you guys, I wasn't expecting that. I was expecting Codeex to have built a better one just based on the way that I've been

12:25 using Codex and Cloud Code and how much I've been using Codex lately. I was honestly expecting Codeex to win this challenge, but so far when we're just looking at the actual like output so

12:34 far, I think Cloud Code is taking the cake. Obviously, that's not to say that Cloud Code's just better write out because it has to do a lot with this prompt. And I'm going to explain what I

12:43 mean by that as we keep digging in here. But anyways, let's take a look at the cost. Cloud Code costed me, if I was using API billing, $832 bucks, which means that Codex costed almost $3,000,

12:53 which is just insane. When it comes to the output tokens, Claude Code used a little over 2 million while Codex used almost 11.5 million output tokens. Now, when it comes to Codeex, it used GBD 5.6

13:05 Soul. That's what I was using to drive. I was using it on high and it orchestrated all of its sub agents and everything like that to use GBD 5.6 Soul. So, all of this is 5.6 Soul. with

13:15 Claude, it did this breakdown where it had Fable 5 going, it had Opus 4.8 going, and it had Opus 5 going. Now, look at this. You see here that Opus 4.8 was the main orchestrator, which really

13:25 threw me off because I didn't start the session on Opus 4.8. I started the session on Fable 5 and I gave that SL goal. I think that it for some reason had some sort of security safeguard

13:34 check and it reverted back to Opus 4.8 and then Opus 4.8 became the main orchestrator and it started spinning up Fable 5 agents to do the work, which I thought was really, really interesting.

13:44 But it was still able to be pretty efficient. I mean, 832 bucks with 2 million output tokens compared to what Codex did over here with 11 million output tokens. Man, that's really

13:54 interesting. Time-wise, Cloud Code took 5 and a half hours while Codex took 61 hours, almost 62 hours. So, that's like 2 and 1/2 days. That I was genuinely shocked when Cloud Code was like, "Hey,

14:04 I'm done." And I had been running Codex for almost a day and a half already, you know, because I was traveling. I kicked it off on Codeex. I was like, "Oh, when I get home, I'm going to send the same

14:12 prompt off to Cloud Code and just see like, you know, how they compare." I was really shocked to see how fast Cloud Code finished here. It's interesting because in previous testing, I've always

14:20 felt like Codeex was a bit better with being efficient and being quick, but obviously this revealed very different things. And I think it has to do with the way I prompted once again. So, we'll

14:29 dig into that in a sec. What did the shape look like? All right. So, now how about the shape? Well, Claude Code did this with one orchestrator, 35 sub aents, and about 2,800 tool calls.

14:38 Whereas Codex did this with one orchestrator, 126 sub aents, and 32.5K tool calls. So I we're kind of understanding the gist of what happened here. Codex worked longer. It used more

14:49 agents. It used more tokens. It used more tools. And Cloud Code did less. And honestly, I think it did better so far. Now, what's really interesting is the tests. We clearly saw bugs in both of

14:59 them, which was a little bit disappointing. But this tells us a lot about the way that these models work. So, Claude Code did 296 unit tests, 199 test cases, and 102 browser tests.

15:11 Whereas Codex did 2,300 unit tests, 341 test cases, and 391 browser tests. So, I've always felt like Claude Code, or I guess Fable is kind of the wise owl. I like to use it for being creative, for

15:25 planning, for brainstorming, for helping me figure out the path. Whereas Codex has never felt that good at that for me. For codeex, it feels like the just it's going to be obedient. It's going to do

15:36 what you say and it's going to do it well. It's going to run tests and it's going to make sure that the job has been done. Which means for me when I prompt cloud code, I like to give it a prompt

15:44 like this. I give it a highle goal. I say, "Hey, this is what I want. This is what good looks like. This is when you stop." And with Codex, it almost feels like you need to be a little bit more

15:52 specific. You need to be more like here's kind of like step one, step two, step three, step four. I just made a video about getting out of the model's way. And that video was based on a talk

16:00 that Boris Churnney had done. And obviously Boris Churnney is the creator of Cloud Code. But clearly in this experiment with codeex, it just didn't seem to interpret what I meant well

16:09 enough. And it didn't seem to be creative enough to explore enough to figure out what sort of experience I was looking for at the end of the goal prompt. Even though it worked so hard

16:19 and so long, I obviously feel like this was a waste of money here. So I thought that was really interesting. And then what I did is I basically inspected the entire sessions. I inspected everything

16:26 they did and I consolidated all that. And then I had Codeex look at both of those and tell me which agent did better. And agent A is cloud code in this case cuz I I kept this anonymous.

16:37 Codeex said that Cloud Code did better which was really interesting. Now before I dig into this guys, I'm not trying to bash on either one of these tools. I use them both on the daily. I will be honest

16:48 with you guys lately for knowledge work. I've been using Codex to drive my sessions. I've been using Codex probably 80% of the time and cloud code probably 20% of the time. But I think it's really

16:56 important for you guys to realize that I still like them both because I use them both for different scenarios and I like to as the models improve, as new updates come out, I switch it around a lot. I'm

17:05 not just going to choose one and say, "Hey, this is my driver for the rest of my life." You know what I mean? So anyways, let's take a look at these categories that Codeex said that Cloud

17:12 Code won in. So product judgment and scope agent a cloud code one with judgment and scope. And that aligns with the way that I feel about it. Cloud Code made clearer must/deer decisions and

17:22 focused on valuable differentiators while Codeex pursued 135 capabilities including several expensive operational features with less restraint. And I feel like that's exactly what we saw. Codex's

17:34 version was overwhelming, wasn't thinking about the user, wasn't thinking about the experience. Codex was just building just to build and and it just was too much. Then the next one,

17:41 architecture and execution. This one went to Codeex. Claude Coat's contract first waves produced zero merge conflicts, but Codeex built the more operationally mature system with

17:51 immutable revisions, offline recovery, migration safety, concurrency handling, and cloud boundaries. So maybe the backend at scale, the infrastructure Codex was building was much better. And

18:01 that's why I love to do a lot of development and planning with cloud code and I love to do security reviews, bug finds, bug fixes, all those types of things with Codeex. You guys have

18:10 probably seen the codeex plugin for cloud code where you run the adversarial review and it's really really helpful and it almost always finds things that my cloud code workflow missed out on

18:19 bugs edge cases things like that. Now the next category testing and reliability codeex one by a wide margin. Cloud code performed strong security and data correctness testing but Codex added

18:29 cross browser testing property tests fault injections blah blah blah. Codex was able to do things like it tested on so many different types of browsers. to test it on mobile and we did not see

18:38 that happening with cloud code as you can see here with the tests. There are just obviously significantly more tests that were being run by codeex and yes it worked for a lot longer and spent more

18:47 money but still the model harness in this case seemed to just do better with the testing. Now I guess some of you could argue like okay well maybe cloud code needed less tests because it did a

18:57 better job building in the first place and that is also a valid argument but I'm just trying to show you what I found here. And the last one here was basically around efficiency. Cloud code

19:05 got a 9.8 out of 10 while Codex got a 5.5 out of 10. Cloud Code finished in 5 12 hours for roughly $447 using 35 agents. See, I know that I said earlier 832. There was a mismatch somewhere. The

19:16 point being Cloud Code spent a lot less. I inspected the session logs and I was mainly for the most part getting this answer. So, I think that this must have been a little bit of a hallucination

19:25 somewhere along the way. I'm not sure where, but according to the slash usage stats of the session and all the sub aents, the number was more like 800. But anyways, Codex took two and a half days

19:36 and spent way more money, way more sub agents. So, Cloud Code did this about 11 times faster and 6.6 times cheaper. And that once again kind of goes against what I thought was going to happen

19:47 because a lot of my tests in the past when I've done Cloud Code versus Codex, Codex has been more efficient with tokens and has also been quicker. But, you know, 5.6 Soul is new. You've got

19:57 Fable over here. You've got Opus 4.8. You just there's so many different variables and it's always like you're pulling a lever at a slot machine. you just don't know what you're going to get

20:04 with these models. But anyways, I hope that this experiment was insightful to you guys. I hope that this at least made you think about the way that you think about these two model harnesses and the

20:12 way that you think about prompting these things cuz it's always changing. And that's why it's so important to kind of be hands-on doing little experiments like this because you never know how

20:22 it's going to fit into your workflow. I think that it's great to be following along with Boris Churnney and Andre Kaparthy and all of these thought leaders in the space and people that are

20:28 actually developing these tools, but something I said in my previous video was like you should not be just taking their advice and blindly applying it because they do different things with

20:36 it. They have different motivations. It would be like if you're a professional triple jumper and you're taking advice on your jumping from a high jumper. Like maybe there are some similarities there

20:44 and maybe the fundamentals and you know some of the foundational things are consistent, but like at the end of the day, it's a completely different sport. It's just a completely different

20:51 ballgame and you probably want to be taking advice from people that are also triple jumpers. That's a real sport, right? Okay. Yeah. So, I knew that this was obviously like in track and field,

20:59 but I just wanted to make sure that it triple jumpers sounds like a weird term. But anyways, guys, that is going to do it for today. So, if you enjoyed the video, you're learning the new, please

21:07 give it a like. It helps me out a ton. And as always, I appreciate you guys making it to the end of the video. And I will see you all in the next one. Thanks everyone.