Hugging Face Trained an Open Model Inside Claude Code

0:00 The same model with the same weights solved 62% of tasks in one app and only 33% in Claude code. Nothing changed except the app around the model. If you pick a model from a leaderboard, that

0:12 app is part of the score. Claude didn't take part in this test. The model is LFM 2.5 from Liquid AI and it has 2.6 billion parameters. Claude code was only the app that the small model ran inside.

0:25 How do you train a model inside Claude code when you can't change any of its code? Hugging Face did it with one extra piece of software. The training gained a lot, but it also cost something and one

0:35 run can't prove everything. Hugging Face's CEO, Clement Delangue, said the team turned Claude code, Codex, Open Code and other coding apps into reinforcement learning RL environments.

0:46 He said they changed neither the apps nor the training code. In an RL environment, the model tries a task and gets a score as its reward. People call these apps harnesses, but I'll call them

0:57 apps. Here's the base model and four apps. Each app got 250 tasks that the model never saw in training. In Open Code, it solved 33.6% and in Claude code, 33.2%.

1:10 In Codex, it solved 40% and in Mini Sweep Agent, 62.1%. Mini Sweep Agent gave this model its best score. The tasks are data analysis problems

1:20 from a set called small data in apps. The team trained on 1,000 tasks and kept 250 for testing. To train a model, the trainer needs to run the task itself. The trainer is the program that updates

1:31 the model. It builds each prompt, calls the model and reads each answer. A real app builds its own prompts and calls the model on its own. So, the trainer sees only the requests

1:42 and the responses that pass between the app and the model. The fix is a capture proxy. It acts as the model's API, so the app sends its requests to the proxy as if the proxy were a normal model

1:52 service. Think of a translator who also keeps a notebook. The app talks to the translator and the translator talks to the real model and writes everything down.

2:01 The first step is a session. Before a run starts, a post request to the proxy sessions endpoint creates a capture session ID. The app gets that ID where its API key normally goes. Every request

2:12 carries the ID, so the proxy knows which run each call belongs to. The write-up doesn't say how the app learns the proxy's address. Second, the proxy finds the dialect of the request. Each app

2:22 sends requests in one of four API formats. I'll call them dialects. OpenAI chat completions, OpenAI responses, Anthropic messages, and Google Gemini. The proxy

2:33 checks the path first, then the headers, then the shape of the body. Third, the proxy converts the request to the chat completions format and sends it to the inference engine that serves the

2:43 model. So, four dialects become one. The model never needs to know which app called. The fourth step is the important one. The proxy asks the engine for three extra things: prompt token IDs, sample

2:55 token IDs, and each sample token's log probability. A token is a chunk of text and the model picks one token at a time. The proxy records all of this for the trainer.

3:06 It also turns off top P and top K, so the model samples from its full distribution. Last comes the reply. The proxy waits for the whole response and stores it. Then it replays the answer in

3:16 the app's own dialect as a stream if the app asked for a stream. So, the app gets an answer in the dialect it expects. These five steps are the authors' own description of their proxy. The finer

3:26 details, like streaming edge cases, rest on that description. Why record tokens and not just text? Different token sequences can produce the same text. If you tokenize the saved text again, you

3:37 can get different IDs from the ones the model sampled. Then the trainer would learn from tokens the model never chose. The same problem applies to the probabilities. For each token, the

3:46 trainer compares two probabilities. One is the probability when the model chose the token, and the other is the probability under its current weight. If you recompute the first one later, it

3:56 uses the updated weight, so the comparison is wrong. The translator picture breaks here because a notebook of words isn't enough. The proxy writes down exact token IDs and probabilities

4:06 while the model generates. A rollout is one full attempt at a task, and it holds many model calls. So, the proxy links the calls into a graph. Each call points to the earlier call whose

4:16 prompt plus completion is the longest exact token prefix of its own prompt. For example, if call two starts with call one's prompt and answer, call two points back to call one. That's how the

4:27 proxy rebuilds the order of the run. Retries become siblings of the call they repeat. Sub-agents, which use a different system prompt, start a new root. Context compaction, where the app

4:38 shortens its history, also starts a new root. Each path from a root to a leaf becomes one training sequence. Context tokens get a loss mask of zero, so the model doesn't learn from them. Tokens

4:49 the model sampled get a mask of one. A turn that doesn't line up stays as context only. Claude code rewrites its history, so one rollout there gives about eight training sequences. That's

4:59 why training in all four apps processed 451 million tokens against 162 million for open code only training. Here's the full path. The app's code stays unchanged, and the session ID goes where

5:11 its API key normally goes. The proxy answers as the model and records every call. Later, each path of calls becomes one training sequence with a mask. The capture idea isn't new. Nvidia's Polar

5:22 paper from the 22nd of May already proxied model calls from apps whose code it can't change. It rebuilt the exact token record of each run. It reported a gain of 22.6 points for Quinn 3.5 4B on

5:36 the SWE-bench verified coding benchmark with the Codex app. The proxy's converters are adapted from Polar's gateway. The new part here is the open packaging that works with many apps. The

5:46 trainer uses GRPO, group relative policy optimization from the DeepSeek math paper. For each task, it samples a group of eight rollouts and scores each one against the others in its group. If

5:57 every rollout is equally right or equally wrong, the group teaches nothing. When one rollout beats the group average, the trainer makes that rollout sample tokens more likely. The

6:07 reward is one for a correct answer plus a bonus of up to 0.1 for fewer tool calls. A wrong answer never earns the bonus. In 22.6% of four app training groups, every scorable rollout was

6:19 correct, but the tool counts differed. In those groups, the bonus was the only signal. That 22.6% has nothing to do with the earlier 22.6 points. Without that bonus, between 35

6:31 and 58% of training steps carried no learning signal. A step here is one update of the model's weights, and it can hold many groups. So, a small bonus gives the trainer something to learn

6:41 from. Remember this because the tool call result later depends on it. The setup uses Async GRPO in TRL, the Hugging Face training library. Each run has two H100 GPUs, one for training and

6:54 one for serving the model with vLLM. Rollouts come from weights that are two to four updates old, so the recorded probabilities matter. Each rollout runs in its own E2B sandbox, which is an

7:05 isolated environment, so one attempt can't affect another. Training took 32 hours for OpenCode only and 46 hours for four app training. Here are the results. Training inside

7:16 OpenCode alone raised the OpenCode score from 34 to about 58%. Claude Code went from 33 to 42, and Codex from 40 to 43. So, most of the gain stayed in OpenCode. Training in four apps at once gave a

7:29 more even result. The scores were 50% in OpenCode, 49 in Claude Code, 54 in Codex, and 62 in Mini Swiss Agent. Compared with OpenCode only training, Claude Code gained seven points and

7:42 Codex gained 11. Four-app training also cost something. In Open Code, it lost eight points from 58% to 50% against Open Code only training. The four-app model also processed far more data, so

7:55 the comparison isn't equal. Both facts matter for the headline. Mini Sweet Agent started at 62.1% and finished near 62%, but four-app training lifted Claude Code from 33% to

8:07 49%, so it added about 16 points in Claude Code and none in Mini Sweet Agent. Pass@1 is the share of tasks solved on the first try, averaged over the four apps. It went from 42.2% for

8:20 the base model to 52.3% with Open Code only training and 54.2% with four-app training. The authors say differences within about two points are noise. So, those last two numbers are close to

8:32 a tie. Next, the team compared RL with supervised fine-tuning or SFT, which means copying a bigger model's answers. They gathered 3,189

8:43 successful rollouts from Quinn 3.8 27B across 888 tasks and four apps. SFT on those gave 47.5% for Open Code only and 43.1% for four-app training. Four-app RL scored

8:58 54.6% at step 700 and 54.2% at step 1,000. So, I'll say about 54. RL beat SFT, but the comparison has limits. The team picked the SFT checkpoints on the test set. The authors

9:12 say the methods differ in objective, data, task pool, and compute, so these are observations, not a ranking of methods. Also, four-app SFT dropped Mini Sweet Agent from 62.1 to 45.2%

9:26 and the authors can't explain why. The four-app RL model also used 31% fewer tool calls. That number comes from tasks both the base and trained model solved, and the reward bonus encouraged

9:37 it. Open code only RL cut only 11%. SFT on the teacher's answers cut 24.2% with no bonus at all. So, fewer calls isn't unique to RL. Why does Claude code score so low for the same weights? The

9:51 write-up sites Quai cats analysis, which blames overfitting to one app's format, context layout, and control flow. But, the write-up itself runs no test that isolates the cause. So, prompt size,

10:02 history rewriting, and tool names are only guesses. Every setup ran once with one random seed, which is the starting choice that shapes a run. All the runs use small models between 2 and 2.6

10:14 billion parameters. Two earlier Quen 3.5 2B runs trained on correctness only peaked at 37 and 39.5% and then fell to about 26. Those runs show that a checkpoint picked at its

10:27 peak can overstate the result. The method needs open weights that you serve yourself because the proxy sits in front of your own inference engine. So, it can't intercept hosted API models.

10:38 The authors say larger runs are coming. Here's a separate check. Zhang and colleagues submitted a paper on the 7th of May. It tested GLM 5.1 on 100 tasks from Swebench verified, a set of real

10:50 software bug fixes. Across three app setups, GLM 5.1 scored 52.5, 56.5, and 65.5%. That's a 13-point swing from the app alone. In each of the three setups, the

11:04 three models differed by only three, 2.5, and five points. The authors also say their 7.8 times ratio of app variance to model variance isn't universal. It's a different study with

11:16 different models, so it supports the pattern, but doesn't prove it here. If you read agent leaderboards, ask which app produced the score. In the Zhang study, the app moved results more than

11:26 the model did. If you build agents, test your model inside your own app. If you train models, copy the capture proxy idea. Treat gaps under about two points as noise. Treat the large gaps as real

11:38 signals, but still from single runs. Claude code scores 33% where mini sweet agent scores 62 for the same weights. Which part of the app causes that gap?