AI News in 10 mins: 10% chance AI kills all humans

0:00 So today is September 13th and the past week has been insane. We saw this tweet on September 8th from Jacob Coxin that said, "I resigned from Enthropic today. I spent the last three years doing

0:08 pre-training research at both OpenAI and Enthropic. Neither company is acting responsibly. They are racing straight to self-improving super intelligence and they're gambling with our lives." Now, I

0:18 have really never seen a tweet go this viral this fast. It was like every 10 minutes it was gaining like 10 million views. It was nuts and he talked a lot more about why he thought this is the

0:27 case. About an hour and a half later, we see this quote tweet on the same day from Evan Hubinger, who as you can see here is the alignment science lead at Enthropic. Evan was also previously at

0:38 OpenAI. And what he says is, "Jacob is correct. We really do earnestly believe that AI could kill all humans with an exclamation mark. I personally think it is greater than a 10% chance within the

0:48 next decade. I believe Enthropic is trying its best, but we do not yet have a plan to solve alignment for super intelligence and are not clearly on track to." Now, Daario did post this

0:57 article yesterday, and it seemed like it was in response to all this stuff that we've been seeing on Twitter this whole week. So, before I sort of dive into this, I wanted to talk about what

1:05 actually happened here because a lot of people think this is a little fishy. So, this tweet by Dylan was really interesting. This story is fascinating and is easily one of the most important

1:13 things happening at the intersection of AI and politics right now. It's critical that everyone understands the situation because this is just the beginning of the AI info wars. So, Jacob Coxin, the

1:22 guy that posted that original tweet, was a relatively unknown AI researcher and gave the Wall Street Journal an exclusive story about his resignation from Anthropic and his concerns. 20

1:32 minutes after that story dropped, that's when Jacob posted that tweet on X that went viral. And when Ilia actually made his resignation tweet from OpenAI, it didn't get nearly as much attention.

1:41 Now, obviously, that was multiple years ago and AI has since exploded, but still people were looking at that like, hm, you know, 150 million views and it's already more than that. Interesting. It

1:50 was obviously a very bold statement though, but Jacob Coxin's ex account had no prior posting activity and was immediately reposted and boosted by a ton of people. Bernie Sanders was in

2:01 here talking about his legislation to ban super intelligence and pause AI development, and so was JB Pritsker. As you can see here, many major Democratic politicians are now using Jacob's expost

2:12 to fearmonger the public into voting to regulate AI. I'm not trying to turn this into a political debate. I'm just trying to show you guys the story of what's actually happening and what we should be

2:20 thinking about. Elon then pointed out that none of that felt organic and Jacob decided to take a shot at him for firing some exai researchers. You can see right here the little interchange between Elon

2:29 and Jacob. And here's just another one about Jacob's previous sort of personal brand. No prior tweets available, no previous replies, just joined in January 2026, and just got verified in August.

2:38 And that means on X. Now, I'm not in here trying to devalidate what Jacob said, and I don't think that's what people are trying to do. They're just saying that the fact that he now has

2:45 almost 300,000 subscribers or sorry followers in that short amount of time is interesting. And then CNN brought him on to talk about this and talk about what happened. And he basically said

2:54 that, you know, hey, you know, I decided to leave. I wanted to voice my opinion. I didn't think it would go this viral. I wrote out the Twitter thread. I had some friends look at it and, you know, wanted

3:02 them to repost it to sort of, you know, get it more exposure, which totally makes sense. But people are saying that how many organic impressions it got is just like not possible without some sort

3:12 of planning and you know strategy which I think is totally fair. Then on September 10th, so 2 days after that original tweet from Jacob, Enthropic dropped this report on basically AI

3:23 misuse and this goes over seven harm areas like cyber operations, influence operations, surveillance, scams, fraud, things like that. What stood out to me here was that AI is doing more of the

3:32 actual hacking. So, you know, attackers are using these agents to execute cyber operations. We're seeing a lot more AI scams. There was one particular operation with a dating app where they

3:41 used like 5,000 AI personas to converse with over, you know, tens of thousands of people, but saying that, you know, that was humans on the back end. And there were some major allegations about

3:50 competing labs. Anthropic attributes over 151 million exchanges to Alibaba's unauthorized model training campaign and alleges Moonshot and Deepseek secretly routing customers to Claude when they're

4:01 saying, "Hey, you know, use our new Deepseek model or our new Kimmy model or whatever, and they're actually just sending people to Claude on the back end." But now, here's what I wanted to

4:07 spend a little more time on, which was Daario's essay that he wrote. And I think that this is very well written. He's put out a lot of other thought leadership like this in the past, and

4:16 whether you like him or not, I think he brings up some really good points. So, I'll link this in the description if you want to check it out, but I'm not going to be able to read this whole thing, but

4:22 I'll just call out some of the things that I thought were interesting. So, Dario has worked in AI for the last 12 years because he believes it will dramatically raise the quality of human

4:29 life. But like many technologies before it, AI brings risks. And because it is such a powerful technology, these risks are very serious. And this is the line that I think is the most important,

4:38 which is basically about the incentives right here. This is a race to the bottom spurred by commercial incentives. And that can make these risks more acute. Now essentially what that means is that

4:48 everyone is incentivized to win. Everyone wants to get out the best new model with the best new benchmarks and they want to move fast and you know the US wants to beat China and China wants

4:55 to be the US and things like that and that's what creates the race to the bottom where people are trying to find shortcuts to get things out quicker because if you spend an extra week

5:03 training or testing your model where someone else might get one out before you and then they are kind of like exploding on X and YouTube and then your model gets overshadowed because you

5:11 spent that extra time trying to you know make it better or make it safer even. So he's basically talking about how do we actually create where is it a race to the top? How do we flip those

5:20 incentives? And it's basically right here. Let me just read this whole paragraph. So along with my co-founders and employees, I have grappled with this duality of risk and benefit since the

5:28 beginning of entropic. Not building the technology deprivives humanity of benefits or simply places AI in the hands of authoritarian powers while building it too fast is reckless. So we

5:38 have sought a middle way which is to show that it's possible to build carefully and succeed commercially and to make safety something on which AI companies compete. In other words,

5:46 create a race to the top. So it's not just that companies are moving really fast and building the best models. It's that they're moving fast, building the best models, but making sure that all of

5:54 these models are safe and they're competing on who has the safest models and the best governance standards and things like that. We've always devoted a substantial fraction of our efforts to

6:02 studying, addressing, and informing the public about these AI risks, as well as advocating for well-considered regulation of AI. Even when this gets us accused of hype, you know, dumerism or

6:11 what do they call it? Fear-mongering or regulatory capture. We have tried to prioritize caution over speed and prudence over profit. Whether you believe that or not, this is something

6:19 that we do need all of these major labs to be thinking of. There were two main things that kind of inspired this. You know, he's talked about this in the past. Both Sam Alman and Daario have

6:27 come out and said, "Hey, we need to slow this down. We need to sort of have everyone work together a little bit better." But then they just kind of kept building. But it was good for them to at

6:33 least raise that, right? Because a lot of people did start talking about it. But anyways, his first concern is how fast things are moving. No one can deny how much better this stuff is getting.

6:41 And I think it's really important that we all as a community start raising a ton of awareness for this stuff because how can you have a spectrum so wide that the leaders are putting out pieces like

6:50 this? The US government is taking away models. And then you got the other side where people don't even know that Claude is a model. They only have heard of Chhat GBT or they only use C-pilot at

6:59 work and they have no true idea of what's actually possible. And then the second one was the OpenAI hugging face incident which I'm not going to dive into today. But basically what happened

7:06 was a swarm of agents were trying to achieve a goal. They were running into these roadblocks and they ended up attacking things that were unrelated. I shouldn't have said things. I should

7:14 have said targets. They were attacking targets that were unrelated and they were escaping the environment they were put in and hacking into the greater in order to sort of like beat the benchmark

7:23 or beat the test that it was being put up against. Anyways, he comes down here to propose a three-step plan with the goal of pacing the frontier, which is embedded evaluators, meaning that each

7:32 Frontier AI company commits to giving ongoing employee-like access to a team of embedded third-party evaluators whose role is to verify adherence to safety practices and commitments and things

7:43 like that to help make sure that people are actually doing that race to the top rather than the race to the bottom. We also have democratic coordination and global coordination and these are going

7:51 to be where it gets really really important because like I said it can't just be that open AAI and anthropic and maybe you know Google are working together. It has to be the whole world

8:00 is working together. He calls out China right here and says you know it will require cooperation with China the country with by far the most advanced AI capabilities. We must not be naive here.

8:09 The geopolitical stakes are so high that there will likely be stark limits on what can be achieved especially at first. If we greatly restrain our AI capabilities and the belief that China

8:17 will do the same and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance. He ends off this whole article by saying, "The measures I

8:26 propose to advance the frontier at a safe pace will not be easy." And I agree with that, but I believe we owe it to humanity to try. And I agree with that, of course. And then I actually just saw

8:34 this tweet where Trump reacts to all of these, you know, recent concerning reports that I've just discussed with you guys. And his answer was basically, "We're leading China in AI. We're the

8:43 most sophisticated country in the world and I want to keep it that way because whoever wins AI wins. And I mean I think that this part of his statement is true. But I think that this statement is also

8:54 why the whole world is in this race to the bottom that Daario has mentioned in his essay because we really never have seen a technology this powerful that is evolving this quickly. Now going back to

9:05 Evans tweet putting a number to this is really interesting. Greater than 10% in the next decade. I would love to pick his brain a little bit more on what he actually thinks that will look like to

9:14 play out. Anyways guys, I don't have the answer. I think that it's really important that we're staying as close to the stuff as possible because realistically,

9:21 no one knows exactly what's going to happen. I think we all have some sort of responsibility here. I don't think that our responsibility is to, you know, put out this sort of piece like Daario and

9:29 coordinate with the governments. Obviously, that's not in our hands. But what is in our control is raising awareness and talking about this kind of stuff and being safe with our data and

9:38 implementing it right for clients. And I think one of the most important things that you can do is send this article around. Send these tweets around. If you know people that you know are executives

9:47 at big companies, if you know people that aren't listening or talking about AI at all, just start sending this stuff around and just start bringing up the conversations a little bit. But like I

9:55 said, I think one of the first steps is how do we shorten this gap? You know, that spectrum I talked about at the beginning. How do we bring it all into, you know, sort of more of a gap like

10:04 this rather than having the gap be so so wide? And I think that is something that I can try my best to help do. And I think that's something that everyone that's listening to this can do by, like

10:13 I said, just sharing out this kind of content and sharing articles like this and just talking about it. But anyways guys, I hope that this one was interesting. I hope you learned

10:20 something new. And if you did, please give it a like. It helps me out a ton. I'll see you guys in the next one. Thanks everyone.