AI Can't Think Like Human But..

summarized

TLDR

AI models do not think like humans; they lack the abductive reasoning that humans use to invent explanations from limited data. A DeepMind position paper argues this is a structural limitation, not a scaling issue. However, models like GPT-6 Astra can perform surprisingly well on tasks like Arc AGI-3, but results are highly sensitive to the testing setup, making benchmark numbers unreliable without context.

Key points

A DeepMind position paper titled 'LLMs Can't Jump' argues that models are structurally incapable of abductive reasoning.

GPT-6 Astra scored 99.9% on Arc AGI-3 using OpenAI's own harness but only 62.7% on the standard harness.

The term 'neuron' in AI is misleading; it is a math function, not a biological cell.

Jeffrey Hinton's 2016 prediction that deep learning would replace radiologists within 5 years was wrong; demand increased.

Yann LeCun advises listening to economists rather than AI researchers about AI's impact on jobs.

Tools mentioned

Techniques

  • induction
  • deduction
  • abduction
  • back propagation
  • reinforcement learning from human feedback
  • world models
  • recursive self-improvement
Transcript (captions)

0:00 Same model, same test, same week. Two scores, 99.9% and 62.7. Both numbers are real and both were published this month. The only thing

0:12 that changed between them was who set up the room. That gap is the whole argument about whether these things think. By the end you'll know what actually happens inside a model when it answers you.

0:22 You'll know why that isn't thinking and why calling it dumb odd and incomplete is the same mistake pointed the other way. We'll start somewhere that has nothing to do with computers. In 2018, a

0:33 Marvel film ended on a cliffhanger. Doctor Strange, sworn to guard the stone with his life, hands it to Thanos. The film never says why and the sequel was a year away. That question filled the gap.

0:45 Ask a language model today and you get a fluent, confident, structured answer as long as it's read the plot somewhere. It didn't sit in that theater. It didn't feel the room go quiet. So you've got

0:56 two roads running to one answer, yours and it's. We're going to walk both and the second one is stranger than you'd expect. The human road first. Einstein drew it and he drew it as a loop. On the

1:08 7th of May, 1952, he posted a diagram to his old friend Maurice Solovine. At the bottom is a line marked E for experiences, everything you've ever seen or touched or felt. At the top is A, the

1:22 axioms, the rule you invent that would explain all of it. And between them, one arrow. He labeled it J, the jump. From the axioms you deduce specific predictions, then check those back

1:34 against experience. That part is logic. That part is safe. The jump isn't. Einstein's own line, there's no logical path from E to A, only an intuitive connection and it stays, in his words,

1:46 subject to revocation. So the most important move in science has no algorithm under it. This July, a researcher at Google DeepMind put that same diagram in front of the machines.

1:58 The paper he wrote gives away its ending in the title, LLMs Can't Jump. He splits reasoning three ways: induction, which means spotting the pattern in data, deduction, proving what

2:10 follows from it, and abduction, inventing the explanation that isn't written down yet. Today's models have mastered induction, and they're getting good at deduction. The one that's

2:20 missing is abduction. So, read the sentence that carries the claim. Structurally actually incapable of the abductive jump required to formulate those premises. Structurally. Not yet,

2:31 not with a bigger model, not next year. And the test case Tom Zahavi picks is the strongest one going, general relativity. Einstein got there with barely any data

2:41 to compress, which is exactly the problem. Now, notice what kind of document this is. A position paper is an argument sent to a conference so other researchers can attack it. The proposed

2:52 fix is the interesting half, physically consistent multimodal world models. Give the thing a body, or the nearest thing to one, because that's what you're carrying around. You know what falling

3:03 feels like and the lurch when a lift starts moving, which is where our vocabulary starts lying to us. One set of words gets used for both sides, neurons, memory,

3:13 reasoning, hallucination. A neuron in a model is a math function. You put numbers in, one number comes out. A neuron in your head is a living cell running on electrical signals,

3:25 chemicals, timing, and thousands of connections firing at once. Same word, nowhere near the same object, and the cell is only where it starts. How a memory forms, how one afternoon

3:36 changes you permanently, how chemistry moves your attention around, those are open questions in neuroscience, not settled ones. So, try this one on yourself. At a party, somebody is having

3:47 a great time, then leaves without saying goodbye. Your head hands you three answers before you've asked for one. Tired, awkward, bored. No one gave you a formula for that. You pulled it out of a

3:59 lifetime of leaving parties. The model has no parties. So, how does it get there? During training, it guesses. It might predict that Doctor Strange gave up the stone because he was afraid.

4:10 That's wrong, and the system has to find out how wrong. That measurement has a name, and the name is loss. Then it repairs itself. Now, inside the model sit billions of dials. Think of the knob

4:22 on a gas hob turning one flame up and down. Every dial nudges how information moves through, and we call them parameters. Back propagation starts at the wrong answer and walks backwards

4:33 through all of them. For each dial, it works out which way to turn it and by how much, so the mistake comes out slightly smaller. Then it turns them by microscopic amounts, billions at a time,

4:45 over more text than you could read in a thousand lifetimes, and the loss goes down. That's the whole of learning here, and there's no insight anywhere in that loop. Then people rank the answers.

4:56 Which reply was better? This one or that one? That's reinforcement learning from human feedback, and it's where the voice you recognize comes from. So, when you ask about Doctor Strange, you're

5:06 switching on a pattern. What you live as a feeling, it holds as a position in a space of numbers. At this point, you have to ask, if a system lands on your answer without one of your experiences,

5:17 does the road it took matter to you? So, that is the machinery start to finish. Now, the other side. Because if I stop here, you'll leave with the wrong idea. To call these models dumb autocompletes

5:29 is lazy, and it's wrong. On the 3rd of September, Open AI shipped GPT-6 Astra. The same day, the Arc Prize Foundation published what it scored. Arc AGI-3 isn't a quiz. They drop the model into a

5:43 game and tell it nothing. No rules, no goal, no instructions. It has to poke at the place, watch what happens, and work out how that world behaves. You run the same games yourself if you want, and

5:55 it's worth 10 minutes of your evening. Astra scored 99.9%. Here's the receipt, and here's the line underneath it that the headline skipped. The 99.9 ran on OpenAI's own harness. A

6:09 provider adapter, which means the model's hidden reasoning state is kept alive from one request to the next. On the standard harness, it keeps only the notes it decides to write down for

6:19 itself. Same model, same puzzles, and it scores 62.7. So, nothing about the model changed. The room changed, and the cheaper run is the one that scored higher. $19,000 for the

6:32 99.9, 26,000 for the 62.7. So, when you see a benchmark number in your feed, ask who built the room. That question is worth more than any number

6:42 you read this year. But, don't let that comfort you. The humans in that comparison were paid testers, sat down for 90-minute sessions. Astra used fewer actions than the median one of them on

6:54 96% of levels. About half the moves, on average. And it got there by turning strange environments into compact symbolic models, then writing its own shorthand to keep track of them. That

7:06 isn't answering questions, that's exploring. So, underestimating this is the same error as calling it human, just upside down. Which brings us to the people you're hearing all of it from.

7:18 Jeffrey Hinton won the Nobel Prize in physics for the maths underneath every model in this video. He puts the odds of AI wiping out humanity at 10 to 20% inside 30 years. A

7:30 real person, a real number, and he means it. But, in 2016, the same man said something else. He told hospitals to stop training radiologists because within 5 years deep learning would be

7:42 doing it better. 10 years on, demand for radiologists is up around 26%. Mayo Clinic's radiology staff grew by more than half. The field is short of people, not drowning in them. Hinton has

7:54 since said he got it wrong. The defense is that he had the direction right and the timing badly wrong, and that he meant image analysis, not the whole job. That's fair. Now, hold both of those at

8:06 once. Then there's Yann LeCun, who helped build this field alongside Hinton and disagrees with him about nearly all of it. When Anthropic's Dario Amodei predicted that half of entry-level

8:18 white-collar jobs could disappear, LeCun posted this. Look at who's on the do not listen list. Sam, Yoshua, Jeff, and me. He put himself on it, and that's the most

8:30 useful sentence anyone in this argument has written all year. The alternative he offers is to ask economist, and he names them. People who have spent whole careers measuring what technology

8:41 actually does to work. LeCun has since left Meta and raised over a billion dollars to build world models because he thinks the language model road is a dead end. So, which of them are you

8:52 following? Here's where I land. Today's models don't think the way you do. The jump isn't there, the body isn't there, and underneath all of it, it's arithmetic on dials. That's the verdict,

9:04 and the receipts are that paper and that harness split. But not thinking like you doesn't make it harmless, and it doesn't make it useless. The thing worth watching isn't a chatbot getting

9:14 chattier. It's recursive self-improvement, a system that improves itself with no human left in the loop. Not a model that writes code, a model that rewrites the process which trains

9:25 the next model, then runs it. That one is still unsolved, and if it lands, most of what I just told you expires. A story from July shows you exactly how this gets bent. On the 27th, Ilya Sutskever's

9:38 company announced a deal with Nvidia. The lab got access to Nvidia's newest GPU platform, about 10 times the compute it had before, on a reported $5 billion investment. It traveled around the

9:51 internet as he says super intelligence is solved and they're scaling it now. What he actually said was this. We have research that is worthy of scaling up and having access to a big

10:02 Nvidia computer will let us do so. That's a company saying its research looks promising. It is not a company announcing super intelligence. One sentence, two completely different

10:13 videos, and which one reached you depended on who retold it. So, notice the shape of it. The people selling you fear and the people selling you the future are both selling you something.

10:24 That includes the ones I just quoted. If a model starts driving your computer, that's a very good tool. It still isn't AGI and no one knows when the real one arrives, so before it does, 10 fake ones

10:36 will turn up and die. So, here's what I'd leave you on. If the missing piece really is a body and world models are the fix, does the next generation get the jump or do we

10:46 find out the jump was never one step at all? That's Cloud Coats. Check the harness, not the headline.

Frontier News · by Hyperjump Technology