GPT-6 Astra FINALLY Solves Computer Use

summarized

TLDR

GPT-6 Astra's computer use capability removes the need for APIs or MCPs by directly controlling applications and websites, as demonstrated in five use cases including video editing in Premiere Pro and phone mirroring. The speaker's own decision chart suggests using CLI or MCP when possible for speed and reliability, indicating computer use is primarily a bridge for tasks without programmatic interfaces. This is a notable step toward general-purpose AI agents, though the demonstrations are promotional and not independently verified.

Key points

GPT-6 Astra can control a computer by directly interacting with applications and websites without APIs or MCPs.

The speaker demonstrated editing a video in Premiere Pro by having GPT-6 Astra generate and insert B-roll using Hixel.

GPT-6 Astra can test user journeys by navigating websites and providing feedback, as shown with glider.com.

The tool can control a phone via mirroring to access apps like Instagram that lack APIs.

The speaker recommends using CLI or MCP for tasks that support them, as computer use is slower and more expensive.

Tools mentioned

Techniques

  • computer use
  • MCP (Model Context Protocol)
  • CLI
  • agentic operating system
Transcript (captions)

0:00 GPT can now control your computer like never before. Now, this is artificial general intelligence. We're hearing this from Jensen Puang, founder of Nvidia, as well as the person that even created the

0:12 term itself. And I'm going to show you how you can do this across five use cases that lets you connect to anything um even if it doesn't have an API in MCP and essentially let you have an

0:22 intelligent AI running around on your computer and it does unlock new use cases. Now, what's cool about this is we no longer need APIs or MCPS. Um, it is a creative intelligence on your own

0:34 computer. On top of that, we can do other capabilities that didn't exist before. So, starting off with the very first use case is going to be this, which is turning inspiration into

0:43 design. Now, this here is an example of a website that I use a lot when I'm thinking about my own design inspiration. Unfortunately, they don't have an MCP or an API fancy speak for

0:54 just way to connect to chat GP2. But what you can do on here for example is search for something like I don't know dashboards right and let's say that this is a website that I want to search for

1:03 dashboard inspiration normally I would have to come over and do this manually but because we've got chat GBT browser we no longer have to do that so we come straight over to chat GBT and give it

1:13 the prompt. Hey there I want you to shoot over to this website and find for me three posters that look gorgeous by searching for them. And then once you've done that, I want you to go ahead and

1:22 use my Hicksfield connection to generate three alternatives for something to do with English Springer Spananiels. Okay, say I want to add an example. And then let's come down. I'm going to give it

1:32 here the name of the website, which is going to be save.com. Now, what's great about this is previously I had to go along and do that. Now I can just get chatg to do that for me. Now, it's

1:42 important to understand that this literally enables you to get anything. So I'm on the podcast that I I run with my buddy Nick. Let's say that there's analytics on here that I was really

1:50 interested in getting, right? Maybe I want to see how this is performing. Well, if you don't have access to the YouTube back end, we can now ask chat GPT to do this and it can do it all

1:59 within the browser. It can find the right stats, the right settings without me having to literally touch a single thing. And just like that, we've now got the outcome. And by the way, guys, if

2:08 you haven't grabbed that coffee, grab that beautiful coffee cuz you're going to need it. These levels are only going to get more epic as we go along. So, you definitely want to make sure you stay

2:15 for this. Now, look at this. Also, design looks good. These were the three inspirations and then this is what it gave us. And I'm really maybe it's cuz I have an affection for spring of spinal.

2:24 I think this is incredible. Like if you look at how it's captured the essence born to spring that's probably I think if spring of sponsors had a catchphrase it would be that. And just look at this

2:33 a sunlet scrapbook. There you go. Come down and it grabs it here. And once again this one here the porcelain. Come down and does it. Why is this important? Because on save just as an example you

2:42 can only scroll scroll so far on a free account to access everything. there's no way you can possibly connect it. So, this just unlocks new things for us. Then, this takes us nicely onto level

2:52 two, which is its ability to literally open up work in Premiere Pro or any app on your computer and then also do some incredible things. So, we give it a prompt. Hey there, I want you to open up

3:02 Premier Pro. I want you to load in the video I have I have saved on my desktop for GPT6 Astram. I want you to identify a moment in the first 10 to 15 seconds of the video. I want you to generate and

3:13 insert and create a five second B-roll using Higs full video generation, one of the best models you can find, and then just let me know when that's done so I can review it. Now, what's really cool

3:24 here is I'm essentially telling it to go ahead and edit an app that exists on my computer. Now, as you can see, guys, it's already gone ahead and actually found the GPT6 Astra video. It also

3:34 opened up Premier Pro by itself. It did this entire section itself. This all sounds like I'm speaking Spanish. I'm going to put a link down below for the community where you get all access to

3:42 all my codeex resources. You get access to the entire Agentic operating system as well as a full course on foundations, building beautiful websites and systems that get you light years ahead of

3:53 everybody else. So, I'll put a link down below so you can go ahead and crush that. Now, already it's identified the strongest insert point is it has unlocked genuinely new use cases for me

4:01 and now it's using Seed Dance 2.5 to go ahead and generate that. And if you're not familiar, HFold is basically a platform where you can create images with the best image generation models on

4:10 the planet. Same with videos, everything you want to in a beautiful system over here and you can connect it super duper easy to codeex. As you can see, for example, it even has the images that we

4:18 generated earlier. And to connect it, all you ever do is come down to MCP and CLI. You come down, click on CLI, and just take a screenshot of this or just give URL to Claude. I'll put a link down

4:28 below so you can click that and it'll open up. And then from that, you can literally create custom B-roll and have it edited inside your own timeline. Now, as you can see, guys, this is what it's

4:37 doing whilst it's working. It's now going ahead and finding out exactly where all the footage is. And you can see it. It's got it there available and open on the timeline and it's having a

4:45 look through my computer to find out where the files are. And if you think about it, guys, it literally is one of those moments where you put your hands behind your head because Premier Pro

4:53 doesn't have an integration like this to best my knowledge. And it's literally going ahead and doing it for me sat there at my computer. So the core takeaway is this is that anything it's

5:02 like we will either automate with you or through you. This is what I said when I basically built a saw my last tech startup. The idea we worked with a lot of different companies and not all of

5:10 them had API connections using technology like this. It isn't always necessarily required now because we can just add it in and do any of the stuff that we like. So now I'm going to zoom

5:19 in and see what this has done so you can get a good sense of it. Let's listen to it. But just dropped. I have been playing around with it since I got my hands on it and it has unlocked

5:26 genuinely new use cases for me. Now, instead of showing you how to create it and you now I think like is that good? Actually, it's not that bad. But again, I didn't give it a prompt. I just said,

5:36 "Hey, find something and insert it." If you give it the right systems, you could take this to a completely new level. And now it's all sat there ready for me inside Premiere Pro using this

5:45 incredible technology. You could also do the same thing in Photoshop for color corrections, anything you want to. Level three itself then if we go down and have a look is going to be around about its

5:53 ability to actually test different things. So let's head back over to chatgptt. Hey there. I want you to head over to glider.com in your codeex browser and I want you to just download

6:02 the app. Stop there and just give me any feedback on that user journey. It's one of my tech startups. I just want to know any feedback from a user's perspective. One of the key differences now with this

6:11 computer use is the fact that we can go from building things to actually testing things like humans. And I'll tell you right now, there are so many companies that are crushing it with this. It's

6:21 opening it up instantaneously. Here we go. And we can see the whole UI and user experience. So this is what I use basically to do my speech dictation. It's one of my startups. And we can see

6:30 how chat GBT is going to go ahead and navigate that for us. So we can see the cursor there. It's at the bottom right. It's going to see can it find where the download button is. It asks you to

6:39 create an account. Awesome. And look at this already guys. The first friction point is that the download for free opens an account creation page instead of starting a download. I'm checking

6:47 whether the site offers a direct installer through its help and now it's going through our docs to go ahead and do this. Of course, we could just ask GL directly, hey there, could you go over

6:55 to the homepage glider.com just give me a quick summary. Are you able to download the app for free? So, we've connected the command mode to things like lets you summarize YouTube videos,

7:03 searches the internet, gets you any question about anything. It's really cool. And as you can see, it does explain some of its services, what it can do, and how it works and the

7:10 features that you get. So again, now we have ChatGpt going through the website, giving us direct user feedback. And you could do this if you built an app and you want to test it in different

7:19 dimensions. ChatGpt computer control can do that. It's something that we couldn't really do before cuz it would just took way too long. I remember I started a video with computer use with no

7:28 mustache. When it was done, I had a mustache. That's how long it used to take. This is lightning fast in comparison. Then this takes us nicely on to level four, which is the fact that we

7:37 can basically get it to test websites. So, this is a website that I built in my last video, which I think looks freaking gorgeous. But let's say I want to go ahead and I want to build I get a new

7:47 image down there or something interesting next to Jack AI. Well, all I can do is grab the URL and head back over to Chat GP2 and I drop a prompt. I'm going to say, "Hey there, this is a

7:55 website I've got. I want you to have a quick look through that website and just find a way that you could integrate and create a image of me. Maybe grab my profile photo. get me a different ver

8:04 version with variation with a different background and just have that down there at the bottom in the footer and integrate it into the website and obviously you can see the website so you

8:11 can give me some dynamic feedback on where you think that is best placed and chatbt becomes most powerful when you stack its capabilities it's computer use as well as its ability to go ahead and

8:22 generate things and review things and again it's a genuinely very generally powerful model the AI is a definition I've always heard AI and I thought oh gosh like it sounds kind of like quite

8:33 futuristic and is that really true? It just means that it can do most things a human can do at a computer and we're at the stage where it can go ahead and do that. And so now chat has finished. We

8:42 can scroll down to the bottom of this website. And as you can see at the bottom left, let me bring it over. It's actually added a beautiful photo and changed the background using the Hixel

8:52 image generator. Really freaking crisp and it's in line with the design cadence of the actual website. Now, if you're wondering, for example, how do you actually manage your usage? I'm going to

9:00 give you I want to touch on that very quickly before we go into level five. So the thing to understand about your usage budget is that it depends on the model, the reasoning, the context and the

9:09 tools. So a short prompt can still trigger quite a long expensive workflow. One of the biggest things that you can do is just make sure that once you complete a task in a window, just open

9:18 up a new one. That way you can get way more usage um instead of actually kind of like wasting all the tokens. And I'm also gonna explain very critically when you should and should not use this to

9:27 save your tokens and really increase your performance. Level five is the ability to control your phone. Now, I want to give a huge shout out to Mark Kf, my buddy, the man, the mythical

9:36 legend, who literally covered this recently, and I was so impressed with it. I was like, I have to include it. So, thank you, Mark, for trailblazing and discovering this, brother. You're

9:43 awesome. Now, what we can actually do is get chat GPT to control our phone using an inbuilt app. So, for example, I could say, "Hey, using the phone mirroring app, I want you to go ahead and open

9:54 Instagram and could you just, for example, check out my analytics for the last month and just tell me any observations or insights that you find about how I could potentially improve."

10:03 Now, what's cool is Instagram itself may not necessarily expose an API to show you your stats or how many people are watching at 3 seconds, but we can use chat GPT here to actually go ahead and

10:16 look inside the app. Again, the general theme is where there is no API, we can build and work through it. As you can see, it's now connecting to my 15 Pro Max and it's looking at some comments

10:25 that we have on my YouTube where this guy is talking about Nvidia. So, you can now go ahead and navigate through everything. And just like that, it's opened up Instagram, which is cool. So,

10:33 it's going to go through now and check out some of my settings. And look at this. Say, for example, you didn't know like where certain things were or like how things works. It can just do that.

10:40 Like, hey, go find me the settings that appeals to things or does whatever you want. And you'll also notice just how fast this is compared to previous generations. It is awesome. So it's

10:49 really now you could set up a workflow for this and then turn this into a skill. So it could be something that you could use replicably whenever you want to. Then the big question is when should

10:56 I use it? I built this handy chart for you. Essentially should the task be automated? Remember the most important question to ask is should I be doing this in the first place? If the answer

11:04 is yes, we go to the next section which is can a CLI or MCP handle it? If it can't do that, it's faster, it's more reliable, generally it's cheaper. If it can't and it needs to be done, then we

11:15 can go ahead and use this to basically bridge the gap of the interface. Now, we've covered the computer use. There's one thing that we have to talk about and that's exactly how to get the most out

11:24 of chat GPT using an Aentic operating system which we're going to do together inside this video right

Frontier News · by Hyperjump Technology