CodeRabbit Hands-On Demo – Deploying a NEW AI Benchmark Site!

0:00 So, basically just like that, right as we refreshed, we now have the site is alive and it has picked up some of the games or all of the games. Today, we're going to be taking a look at Code Rabbit

0:09 and a very interesting feature that they have introduced called the change stack. Now, to give a little bit of pertinent information about what we're going to be seeing in today's video, we're

0:18 essentially going to be doing a lot of stuff related to large code base changes, things like pull requests, merges, and looking at this, which is basically a tool to really simplify the

0:29 process of actually understanding how to do PRs, merges, and everything like that in a world where so many folks are now becoming capable of actually building applications. It can be intimidating to

0:41 find yourself in GitHub in the ecosystem and seeing, "Whoa, I don't really understand a lot of this. The diffs look scary." And it is a barrier to entry that Code Rabbit, in specific this

0:51 change stack, really attacks head-on to massively simplify the understanding for folks. And overall, it just really speeds up the process to a high degree. So, for a bit of pertinent information

1:02 beyond what we see right here, which we will get into more intricately later on in the video, we are going to be changing the Ultimate Play website that I myself have made where we post some of

1:12 our game designs and things like that. Single file HTML games from the videos live on here. Additionally to that, I have taken to heart some of the specific feedback that comes from video comments

1:23 saying, "Hey, maybe you should have like a benchmark for a one-stop shop for your thoughts on some of the best models and their specific scores and some of the tasks we like to do, like the browser

1:33 OS, Subway game, etc." So, the culmination of today's video is actually going to be pushing, I think 12,000 lines of AI-generated code live, and then seeing how Code Rabbit can help us

1:42 with the change stack actually just ensure we don't mess anything up, being that this is going straight to deployment by the end of this video. So, before we get into it, please do feel

1:51 free to subscribe, and thank you to Code Rabbit for sponsoring today's video. I've seen they work with some pretty beefy names in terms of like YouTubers, so I was very excited that they wanted

2:00 to also work with me. I appreciate that and essentially it allows me to do more fun things like these specific websites and hosting them. It is a little pricey to host the entirety of the site that I

2:09 have just with some of the back-end stuff and storage and things of the sort. So with that, our next step now is essentially going to be to dump the entirety of this redesign into GitHub

2:18 and then Code Rabbit will come in play and we're basically going to get a good hands-on like diving in headfirst view of how this is going to assist us. So our first step is essentially to push

2:28 these changes onto our GitHub, the repository that I do have for the site. We can see here just with a swipe of the trackpad on this Mac. I don't normally use Mac in videos, but because I have

2:38 been designing the updates for the site on this along with the Claude desktop app, it is more fitting to just do it natively on this system. We can see here that I have been using Fable 5.

2:48 Additionally to that, there is going to be about 12,000 lines of code here that are just going to be uploaded and replacing pretty much everything for this site in this GitHub repository. So

2:57 as I press enter right here, telling Claude to push the changes, we're going to have to wait a little bit of time for that to happen and then when we swap over to GitHub to see this pull request,

3:06 we're going to begin to start to see some of what Code Rabbit does and why it is helpful, especially when you're making big changes like this where I didn't read all these lines of code to

3:15 be completely honest. And we can see right here pushing the V2 redesign. So this is genuinely something that's been a long time in the making. This isn't some like randomly generated code base

3:24 to just make a video around. And folks, if you watch the channel or have for a bit of time, you're likely at least somewhat familiar with the existence of this site. So thank you for that and I

3:33 am excited to just like get it to a better looking point. And then additionally, there's going to be a hardware component to it because as a content creator, you eventually feel

3:42 compelled to try to sell things to your audience. So I'm I'm like 94% kidding, but I am actually designing a cheap cheap cheap little like gaming console that ports

3:53 these HTML games to it and runs on like very cheap hardware. So, it's not something that's going to be ready anytime soon, but there are placeholders worded on the site because I did like

4:02 some of the art that it added to the home page. All right, I just had to do one authentication thing for Fable to be able to push this to GitHub from this Mac, which I have done as I told it

4:11 done, friend, and now it's going to push it. So, now that the pull request has been opened in the ultimate play repository, which is just where all of the files for this site get hosted,

4:20 we're going to see quite a few things here, just some related to Vercel, etc. But, if we scroll down a bit more, now is where we see Code Rabbit AI. The bot has commented and it says it's currently

4:29 processing new changes in this PR. This may take a few minutes, so please wait. And once that has completed, this is going to change, so we will actually have a button appear here, which will

4:39 say something like change stack. And once we click that, we're basically going to go into an entirely dedicated page for this specific pull request, and we'll get a bunch more information about

4:50 things that may be blockers to a functional deployment, the ability to actually chat with an AI agent about some of these things, big lists of diffs, but in a way that makes it a bit

4:59 simpler for perhaps non-professionals to understand exactly what's going on. So, once this has concluded, we will just hop into that page and see a bit more. So, our next step, now that we can see

5:09 the review change stack button has appeared right here, is to essentially click this and we'll be brought into a dedicated interface right here, entirely about this pull request and all of the

5:18 changes. We can see there is a lot of stuff that appears right here, but one of the cool things that I very much like about this just off the bat, actually there are a couple, and one of these is

5:27 maybe like not as directly related, but I do kind of just want to show it. So, while not directly related, one of the things that I do like is you can actually set the profile

5:37 for the reviewer. In this case, we have it set to chill, however, you can also do quiet or assertive. There are a bunch of other interesting options here. Some of them are kind of almost funny, but

5:48 they're cool to see and then some of them are a bit useful. So, if we scroll down, we can see there's a bunch of things here with check boxes as well as with text boxes if you want to do

5:57 things. One thing that I wanted to show in specific is, let me find it. It's a poem. So, generate a poem for the walk-through comment. So, you can toggle that on or

6:06 off. It is toggled off by default, which I suppose from a business perspective is probably the best move. However, it was cool to see and then additionally to that, if we scroll all the way down, we

6:15 do have this anti-slop feature. So, enable or disable slop in low-quality PR detection. So, there's just some interesting things here in terms of like settings that aren't necessarily

6:25 directly related to what we're going to be doing today, but I wanted to just mention them as I think they're kind of cool. So, the first thing I want to take note of is in the left-hand pane right

6:34 here where we have this overview and it is sorted by layers currently. Now, these layers are essentially groupings of all the specific files that are being changed and the diffs and things of that

6:44 sort actually put into groups depending on where they fit together. So, nothing told it that, "Oh, these need to be grouped together." It essentially looked at all of the proposed changes and then

6:54 grouped all of the files into groups depending on where they sit. Additionally to that, it groups them just based in like almost hierarchy as well where like UI is down here and

7:04 things of the sort. So, this is pretty interesting and in lieu of perhaps a messier way of looking at these, which we may be more familiar with if we just click on the files button right here and

7:13 we see them all kind of arranged in a more traditional way if we don't have something that puts them all into a group. This just makes it a bit simpler to kind of handle and make sense and

7:23 intake what you're actually going to be changing. Another thing we're going to see if we just hover over the data contracts and persistent thing is just a little tool tip talking a bit more about

7:32 it and some of these specific stats for each one in terms of like the number of files and things of the sort. Now, another thing we can see is there is actually a diagram right here and it's

7:41 just drawn some of the platform data relationships for this specific site just based off of what it took in from reading over this PR. Now, if we actually look at some of these diffs

7:51 right here, if you're familiar with it, like a normal diff just says like, "Okay, this was changed." This actually tells us what was changed instead of like this line was removed, this line

8:00 was added. So, if we look through, we can see some of the overviews of this depending on how zoomed in the screen is currently. Now, one thing I will note is right here it said dependency from side

8:10 react to stripe. So, in that case, that's not entirely accurate, and that's why it's still good to just read things over manually, but even just putting everything like this together in a

8:19 cleaner way to actually be able to understand with higher-level overviews as well of what these specific diffs are in lieu of just seeing like a bunch of line numbers in red and green and plus

8:28 and minus. And honestly, I really do think that something like this has a place, especially as more folks enter into the world of vibe coding and things like this. Not everyone is going to have

8:38 the ground-up training to understand natively like pull requests, merges, diffs, etc. So, something like this that can actually still teach some good practice and the architectural way in

8:49 which changes should be made in developing software, but can allow someone to have like, "Oh, okay, a more human-readable look at it." I think that's something that is going to become

8:57 more and more important as more folks are able to actually ship software that they may not previously have been able to prior to the advent of like the modern AI systems. Something else that's

9:08 kind of cool is we of course have the number of changes just listed right here, but additionally to that, if we click on this, we can actually see the semantic changes or the semantic

9:17 outline. And these are actually linked to the specific parts where it has identified these. And again, we have to keep in mind that this works for every single specific change in all of these

9:27 layers separately. So, to kind of highlight that, if we close that right there, we can also do mark viewed, and then it will just kind of bring us to the next one. However, what I want to

9:36 showcase right here is if we click this, this is our layer context panel and this is going to essentially show us the specific findings that this has pointed out may be problematic depending on

9:46 their severity. It also has suggestions for them and we can actually click on this to show us a prompt for an AI agent on how to specifically fix this. However, that's not the only thing we

9:57 get from this. So, there are a couple of other things and I need to find one that actually shows this to us. So, as I scroll down right here to basically take a peek at this. Let me expand this out

10:07 just a teeny bit so we can see some more. Now, in some cases we will also have a proposed fix which if we just show in the drop down just shows us basically the differences and the

10:15 proposed fix as it does say in the top header. Now, the thing I want to mention is specifically that this is going to change for every single individual layer that we have here. So, if we click on

10:26 another one, we see that the number of summaries and the number of comments change. The number of severities that are found will change depending on which specific layer we're in. So, this just

10:35 really heavily organizes a lot of this into a pretty manageable front end interface here because we can see this in lieu of just having to try to like comb through it in something like this

10:45 instead. Being able to look at it layer by layer with things that are actually correlated to one another really does simplify a lot of this. Now, additionally in some of these issues

10:53 right here, for example, this one we actually get an analysis chain and if we click a drop down right here, we can see that this actually did execute a script. In this specific case, I believe this

11:03 one is just essentially checking if two people happen to upload the same exact like slug for the game, what will happen and there could be an issue, but because it is so unlikely for the exact same

11:15 thing to be updated within like 200 milliseconds on this side, it's something we can safely kind of just ignore, but we do actually see all of these specific output here and that it

11:24 did actually find that and then run the script just to make note of it, which is kind of cool. So, in the case of the specific issue that we just took a look at with the analysis chain. If we click

11:34 on it right here, we'll actually open it and we can see this happens right here just in this viewer. If we click this reply box right here, we can actually give this context with which it will

11:43 then know to judge future runs through changes in our codebase as well. So, essentially we could give it a context bit basically like, "Yes, this is correct. I'll fix it." However, this

11:53 site gets low enough traffic that this doesn't need to be treated as a major issue. In the future, it can be treated as like or less severe issue. And then it will actually learn based off of our

12:03 specific like style or constraints that a human would need to basically tell it depending on the site's traffic, architecture, etc. And then that just gives a context for handling specific

12:14 things like this in the future and just better tunes it for our specific deployment instance or codebase, etc. Now, another thing that's kind of cool, and again, we kind of look through all

12:23 of these and basically the comments, the summaries of everything, this is true for every single one of these individual layers, which is just pretty darn cool. I think it makes it much easier.

12:32 However, there's something else here that's definitely more in line with like AI native, and that is basically if we go back here into the right-hand menu and we click on the agent button, this

12:42 actually gets us a chat with an AI agent about everything that we see on this specific page. So, in this scope, let me just ask it something that I know the answer to very easily off the top of my

12:53 head. So, I'm asking it Oh, if I can spell right. What happens if the Stripe API key isn't set? Don't worry, I'm I will only ultimately

13:02 implement payment on this site if I do sell a hardware device through it as well that will be open source. So, that's the only reason for mention of Stripe at all. But I'm asking it, "What

13:11 happens if the Stripe API key isn't set?" Because I'm nowhere near done with my hardware project, it's fine. It's designed to not need to be set. So, that's essentially the answer that we're

13:20 going to want to get for that query here, but it just shows us that we can from within this interface just chat with the agent about everything that we see right here in specific emphasis like

13:29 before pushing these live. So, we get our answer. If the Stripe secret key is not set, the app degrades gracefully rather than crashing. So, essentially for a TLDR of this issue

13:40 right here, but we do have citations where it specifically formed its answer from looking through all of these changes. So, missing the key simply disables the store checkout flow 503

13:49 response while the rest of the app functions normally. So, that is the correct answer for this and just a simple question that I know the answer to pretty easily off the top of my head

13:58 as I don't want that feature enabled right now, but I just want it there in case. So, as we get closer to essentially actually pushing some of these things live, well, we'll fix them

14:06 first. One thing I want to highlight as well just before we get into like the next steps. If we look at the overview section right here, we see a pretty decent overview of everything that's

14:16 been done. It does also generate the summary of these changes. We have a walk-through of the changes as well, which does also show us some diagrams and things of that sort as well. And

14:25 then just a bunch of top-level but easy to ingest things here such as high priority concerns, doesn't block merge but may warrant attention, and then these are all of course hyperlinks. So,

14:35 if we click on view comment right there, we get a bunch of additional pertinent information designed to just help us kind of fix this such as the prompt for AI agents that we could then just copy

14:44 and paste back into Claude or something of the sort. And then of course, we have the option to reply as well if we want to tell it like, "Oh, no, this isn't an issue in this specific repository." The

14:53 final thing I'm going to do from within the change stack UI here is just in the submit review pane. I'm going to leave a comment just overall like summarizing the things we found here. And then once

15:03 that's done, I am just going to click submit review. So, our next step is going to go back right here to the specific pull request. We see we have a summary by Code Rabbit, and then if we

15:11 scroll down a bit, we basically have the option to just one-click fix all of the issues that it found. We have the auto-fix drop-down which I have selected right here, and we can either opt to

15:21 push a commit to this branch, which is recommended, or create a new PR with the fixes. In this case, I think I am going to opt to just go with the recommended one, which is to

15:30 basically fix everything in one click, and I would imagine this will take a little bit of time. And after a bit of time, we can see that the auto fix status has changed to completed just as

15:39 denoted by the green check mark. All right, so we're basically ready to merge now. The only thing here is this Vercel failure, which we can ignore because it just refused to build a preview from a

15:48 machine it didn't recognize. Once I push this from my own GitHub account, it will be totally fine. So, we're going to click merge full request right here. We're going to confirm merge, and then

15:58 once this is done, basically, probably pretty quickly, once we refresh the site a few times, the new version will be live. And then, basically, we just did an actual like live deployment of a

16:08 real thing, which was the whole point of like the excitement. All right. So, basically, just like that, right as we refreshed, we now have the site is alive, and it has picked up

16:19 some of the games or all of the games that were already existing on the site because they were just saved in the database in the back end. So, basically, all of the games that had been uploaded

16:29 here are still here, which I am excited about. Now, of course, there will still be some changes that have to be made for this just in terms of QA and making the models up-to-date and things like this.

16:39 However, if we click on one of them, such as the low-poly demo derby that Fable 5 had made just from a reference photo, it still does work, and that is pretty darn cool. So, we have upvote, we

16:49 have the ability to remix the prompt. And like, these are some things that we're going to have to get sorted. But really, the whole big draw of this is we just migrated, or not migrated, but we

16:58 made some pretty significant changes to a site that had a lot of things in the back end, and basically, thanks to Code Rabbit, we're able to pretty simply understand a lot of what was going on

17:09 here. And that's basically, so you're viewing an older snapshot. Comments will be attached to this commit and appear outdated. So, this is probably one of the other things that we want to mention

17:17 is this keeps like a timeline like chronologically of the changes you've made and things like this. So, it can be used again to basically weave a thread through the specific things that you

17:27 have in your repo. Like how it was pointing out if two users upload something and it gets the same slug, this could be an issue. But then I told it, "Well, this is a low traffic site.

17:36 So, we don't have to make that like a priority issue." And it will remember context and you also have like that timeline of changes throughout. And again, if we go back and look at the now

17:45 merged review for the V2 redesign, we see a summary by Code Rabbit and then reviewers Code Rabbit Bot and then all of the changes that were applied, etc. And then again, the site is now live and

17:56 I basically just have to work on actually getting this to be decent. So, for this, I am going to add some entries so we can have some results that were shown in model review videos and then

18:07 play those. But we'll also have a couple of other things such as the CYD console, which is not yet coming, and the join the waitlist button shouldn't do anything. Um, it's possible that it

18:17 does, but it's not necessarily wired up to do anything. So, there's basically going to be a bit of additional work into just getting this to a good point. So, really to close it out, we go back

18:26 to Code Rabbit and how essentially, thanks to the change stack here, we were able to just quickly and in an easy to understand way see a lot of the process that will definitely be scary to

18:37 beginners who don't have as much software experience or any really. Really, I think the thing to kind of drive home and take away is that this makes it much simpler to understand a

18:46 lot of this. Just the way it shows you like the changes and it gives you plain English understanding of these things. It hyperlinks them. The way it basically organizes everything into layers that

18:55 make sense based on how these pieces fit together in your repository. The ability to actually leave comments that will steer it in making future changes. The timeline feature where it understands

19:06 and can reminisce on like previous changes. And then also just the AI chat feature where you can just ask it questions specifically pertaining to these PRs. That's really quite useful

19:16 and it definitely serves a very, very good point in helping folks who may not be as up-to-date in terms of like all of this stuff cuz it's intimidating and this just makes it it's almost like a

19:27 beginner-friendly approach to actual like merging and code review and everything like that. So, that is probably going to conclude today's video where one, we've launched a new site, so

19:38 I am excited about this and we will be able to ensure that this gets updated pretty frequently which I'm going to make darn sure happens. Additionally to that, we got to take a look at a very,

19:47 very cool platform in the form of the Code Rabbit Change Stack. So, this is very cool. Again, thank you to Code Rabbit for basically allowing me to continue putting out videos at a rapid

19:57 pace and then also spending some time and expense on stuff like this which is just kind of fun. It's like a fun little hobby thing to have at least until the CYD display. No. All right. So, that's

20:08 going to wrap it up. If you have any questions on Code Rabbit or the V2 redesign, feel free to leave them in the comments and there will also be a link, a pinned comment just if you want to

20:18 explore this yourself of course. So, that's going to wrap it up. Thanks for watching.