They Made Qwen a Better Agent Without Making It Bigger

0:00 Neelhorse's researchers improved Qwen without moving to a bigger model. On the same coding task, one followed the 5-minute rule. The other used 2 seconds. Can better training make your small

0:12 coding agent more reliable? Follow that bug through the training pipeline and you'll see why it improved and where the gain stopped. The paper compares original Qwen with its trained Neelhorse

0:23 version, both in the 9-billion parameter class. Their assignment was to finish a data auditor inside an existing repository. It had to reject information arriving after a cutoff while allowing

0:34 the read me's 5-minute tolerance. So, a record at 10:04 belongs inside a 10:00 cutoff's allowed window. The researchers report that original Qwen used 2 seconds instead. It rejected that valid record.

0:47 Neelhorse read the read me, applied 5 minutes, and kept it. Same requirement, different result. Original Qwen also changed the output file name in a required field. Its rejection file lost

1:00 the required summary structure. You could get a script that runs, then discover the next program can't consume its output. The job includes the surrounding contract. Neelhorse instead

1:11 kept the command-line interface, executed the requested command, and opened both output files. Valid samples remained in order. Late features appeared in the rejection report. This

1:22 is the researchers recorded case study. I haven't rerun either model on that repository, so we can check the rule itself. My small demonstration uses the paper's 10:00 cutoff and features at

1:34 10:04 and 10:06. 5 minutes of tolerance keeps the first and rejects the second. 2 seconds rejects both. This checks the arithmetic, not the models. Reading the read me sounds ordinary. That's why I

1:47 like this example. A coding agent needs to carry discovered requirements through to a check deliverable. Neelhorse's September 8th paper asks whether those habits can be

1:57 learned from recorded agent interactions without moving to a larger model. Token rhythm releases 4 billion and 9 billion versions trained from the corresponding Qwen 3.5 models. In my comparison of

2:10 their official configurations, the checked language model dimensions match in both pairs, but Neo horse ships text only weights. Original Qwen also includes vision.

2:21 The same size claim concerns text agent performance, not equal capability in every modality. Now return to our auditor. It's complete interaction record is what the paper calls a

2:32 trajectory. The harness manages what the agent can see and do. Training needs that context, and action chosen after reading a file may make no sense without its contents. So, saving only the final

2:44 answer loses the path that produced it. The authors organize those records around user turns. One request can produce several assistant responses with tool results between them.

2:56 Those responses stay together as a training unit. Earlier visible messages and tool interactions remain as context. Earlier reasoning is omitted. That preserves the current request chain of

3:06 action and feedback. First, the reported pipeline checks the structure. Does each tool result belong to a real call? Are identifiers consistent? An intact trace proceeds. A recoverable fragment

3:19 contributes only its complete causal sequence. An ambiguous trace is quarantined. A saved conversation doesn't automatically contain a coherent training example, but did the agent also

3:30 do the job well? The authors assess execution quality, grounding judgments in recorded events. Missing evidence stays missing, and interrupted check doesn't become a pass.

3:42 For our auditor, writing a file and satisfying its required schema are separate observations. Your training pipeline should preserve that distinction. The records also describe

3:52 the user's goal and the observed outcome. That lets the team group problems by capability. A missing readme read differs from a correct implementation saved to the wrong path.

4:03 Keep the evidence that locates the failure. But a final thumbs down tells you much less about what needs teaching. Now for the routing decision. During execution, the harness routes to work

4:14 among models. It keeps predicted demand separate from the service actually used. A powerful model might have been selected because another service was unavailable or the user overrode the

4:25 choice. Its presence alone doesn't prove the task was difficult. Training needs to distinguish those reasons. How do you order those examples for training? The researchers estimate demand from the

4:36 request and history available before the response. Lower demand examples enter earlier. Harder examples enter later. The estimate arranges the curriculum. It doesn't certify each action's

4:48 difficulty and it doesn't change the recorded answer the model is trained to produce. The curriculum has three stages with roughly equal numbers of examples. Some easier material is reserved for

4:59 later stages keeping earlier skills in view. Each example appears once per pass. The optimizer continues across stage boundaries. The model isn't restarted whenever the material gets

5:11 harder. These stages belong to one training pass. Within each example, assistant responses receive prediction loss including assistant tool calls. User messages and tool results supply

5:23 context without receiving that loss. In our example, the assistant learns what to do after seeing the readme. It should not learn to invent the readme's contents. The paper's loss mask makes

5:34 the separation explicit. Recorded examples show another model's actions but your deployed model continues from its own choices. The report describes on policy distillation to address that

5:46 mismatch. The student generates a response. A fixed teacher scores possible next tokens along its path. Training updates the student. The teacher doesn't become part of the

5:56 deployed checkpoint. Routing estimates also organize those starting contexts. Neither serving a request nor scoring its difficulty instantly rewrites the model.

6:06 Instead, weight updates happen through training. The updated checkpoint must return to the harness before its new behavior generates further feedback. That separation matters when you hear

6:16 the phrase recursive self-improvement. Evaluation supplies the next lessons priorities. The authors test the current checkpoint on a separate suite, group its weak

6:26 capabilities, and shift the training mixture toward those regions while preserving broad coverage. Verified successful traces supply examples. Failures identify where suitable

6:36 examples are needed. Failed answers aren't simply copied for imitation. Suppose held out auditor tasks reveal trouble preserving output formats. The next mixture could

6:47 emphasize verified examples that follow the required schema. This illustrates the allocation rule. It isn't a claim that our particular case was added to training.

6:56 Evaluation locates a gap, and candidate examples still face the quality checks. Otherwise, teaching evaluation answers could make memorization look like improvement. The researchers say they

7:08 screen training candidates for exact and near duplicate overlap with evaluation suites. Feedback about weaknesses crosses that boundary. The held out question should remain outside project

7:22 before trusting a rising score. Now the authors compare harness trajectories with a public synthetic agent data set under the same curriculum and closely matched training budgets. The harness

7:33 data checkpoint scores higher on all five compared benchmarks. That supports these execution records within this experiment. It doesn't isolate every ingredient or establish that any

7:44 production logged makes better training data. Across the main 10 benchmarks, the 4 billion models reported average rises from 58.94

7:53 to 64.87, 5.93 score points. Every listed benchmark improves against original Quen. This average combines different tests. It can't tell you the percentage

8:05 of your own jobs the agent will complete. On HumanEval, which checks generated code for functional correctness, the 4 billion score rises from 87.2 to 96.95,

8:17 but a coding test can't establish that the agent will preserve your repository's interface. Our auditor case supplies behavioral evidence that this score alone can't

8:26 provide. The Berkeley function calling score barely moves. 61.02 becomes 61.79. Workbuddies' workplace score improves by almost 10 points. If your workload

8:38 mainly selects and formats tool calls, that larger workplace gain isn't a substitute for checking the smaller tool calling gain. Choose the row that matches the job. These are the Neo Horse

8:48 team's numbers. Their model cards repeat the paper. They aren't independent replications. The authors report matched harnesses and interaction budgets within each benchmark. Original Quen and Neo

9:00 Horse use thinking mode with the same stated sampling settings. Those controls matter more than putting unrelated leaderboard rows side by side. Workbuddy is average over three runs. Some other

9:11 agent benchmarks have only one. That limits confidence in small gaps. For your workflow, repeat task with a fixed harness and budget and inspect the delivered artifacts. Keep failed traces,

9:23 too. A single neat demonstration can't tell you how reliably the model behaves. Upgraded 4 billion Neo Horse approaches original 9 billion Quen's overall average, 64.87

9:35 versus 65.6. But original 9 billion Quen remains ahead on live code bench. Training recovered ground at the smaller scale. That's useful if memory constrains your

9:46 choice, though this paper doesn't establish a universal replacement for larger models. At 9 billion, live code bench stays unchanged at 65.14. Another instruction following test is

9:59 unchanged, too. I eval slips from 89.46 to 89.09 even as the overall average improves. An aggregate win should send you into the relevant rows rather than end the

10:11 investigation. The same unevenness appears in the case studies. In a daily reporting task, upgraded Neo horse gets ticket counts and time-bounded records right according to the authors'

10:22 recomputation. But its status table still uses priority labels where status labels belong. You would still need to validate that report before another system relies on it. Size remains useful

10:33 after training. In the authors' code repair example, trained 4 billion Neo horse stops without an effective repair loop. Trained 9 billion Neo horse uses execution feedback until the verifier

10:45 passes. This compares two trained models, not original versus upgraded Quen. It shows why the title can't become a claim that size never matters. That

10:56 leaves the deepest limit, the feedback loop. The authors explicitly say these results cover a single pass. They have not demonstrated gains accumulating across successive generations. More

11:07 experience could reveal new weaknesses. Whether training keeps converting them into improvement remains an open experiment. A diagram returning to training can't establish that result.

11:18 Neo horse is worth testing if you need a small text-based agent to carry requirements through execution. Targeted post-training improved the reported results without moving up a

11:28 model size class. But scores can stall and outputs remain wrong. I would judge the upgrade on your delivery contract with sustained recursive improvement still unproven.

11:38 Remember the auditor? Did the agent carry the read me rule all the way into a verified output file? Keep that check fixed when you compare checkpoints. Then ask what the failures

11:49 tell you to teach next. That's a practical way to use this paper even while the longer term learning loop remains an experiment.