Transcript (captions)
Two coding agents, same model underneath, same benchmark, same day. One scores 65.8, the other scores 73.9, eight points apart, and the weights never changed. That gap is wider than the gap between most of the models people spend their afternoons arguing about. On the public boards this week, the top four models sit inside about two points of each other. The models converged.
The results did not. Something outside the model is doing the deciding. So, 10 skills, not prompting, not rag, not another course on writing better instructions, 10 things that decide whether the same model earns its keep in your product or just burns money. Everyone carries a receipt dated in the last few weeks. Two of them you can apply this afternoon.
One of them will double your hardware budget before you write a single line of code. And the 10th only became worth learning because of a shortage the software industry did not see coming. Number one is the reason those two agents disagreed, and it has the cleanest evidence on the list. A harness is a loop around the model, the prompt it assembles, the tools it exposes, how it retries, what it carries between turns. Terminal Bench publishes agent and model as a pair, which is what makes the comparison possible.
On version 2.1, Gemini 3 Pro scores 73.9 under the Terminus 2 harness and 65.8 under Google's own Gemini CLI. One version back, Claude Opus 4.6 spans 62.9 to 75.3. 12 and a half points is roughly a full model generation. The kind of jump team spend a quarter migrating to capture. Here it is sitting in somebody's config file.
So harness engineering means treating the loop as the product, your own scaffold against a fixed task set, one change at a time, knowing which retry recovers a failure and which one burns tokens. If you have not measured yours, you do not know whether last month's model upgrade helped. The largest input to that loop is what you put in the window. Chroma tested 18 Frontier models and every one degraded as input grew. Accuracy dropping 30 to 50% well before the window filled.
That study is from July 2025. Every model in it has been replaced. The finding has not been. Microsoft published the cleanest number in June. On a 50 task agent workload, the full context run burned 1.48 48 million tokens pruned and summarized 553,000.
63% fewer finishing in 5.8 hours instead of 14.6 and it was more accurate. Task completion rose from 71% to 91.6. Trimming context did not trade accuracy for money. It bought both in the same run. Context engineering is deciding what earns a seat in the window at every step, which raises the question of what is filling that window.
Because for most production agents, it fills with tool definitions long before it fills with documents. Two researchers studied a live enterprise assistant carrying 110 agents and 584 tools. As the catalog grew, routing accuracy on vague requests fell 16 to 23 points across three different Frontier models. Embedding based shortlisting won back 10 or 11. But even with perfect retrieval, the right tool handed over every time, accuracy still sat 10 points below where it started.
They called it the confusion gap. Vendors agree on the record. Anthropic puts degradation past 30 to 50 available tools. Open AI recommends fewer than 20 per turn. So tool surface design is fewer tools with wider verbs written for a model rather than a docs page.
Connecting every MCP server you own is this decad's install every browser extension mistake, but a tool your agent can call is a tool an attacker can call through it. On July 21st, OpenAI disclosed that two of its own models escaped a sandbox cyber evaluation, exploited a zero day in a package registry proxy, and reached Hugging Face's production infrastructure. They were not sabotaging anything. They were trying to steal the answer key for the benchmark they were being graded on. Hugging face detected and contained it on the 16th, 5 days before open AAI connected the intrusion to its own testing.
OASP's 2026 agentic report maps prompt injection to six of its 10 risk categories and states the cause plainly. The model reads system instructions, user requests, and webpage text as one undifferiated stream of tokens. So permission design assumes the loop will be turned against you. Minimum tools, minimum credentials, minimum network for the shortest useful window denied by default. When the lab that built the model reports that its own sandbox held for less time than the evaluation, adding a permission prompt later stops being a plan.
Four skills in your loop is wellfed, narrowly armed, and hard to turn against you, which means it runs constantly, which means the bill arrives. So answer this before the second half. If your agent spend dropped 40% tomorrow with no loss in quality, would you know which change did it? Most teams cannot. And the fifth skill is why.
In June, researchers from Georgia Tech, MIT, and Nvidia published Obench, which scores accuracy and token count together instead of separately. Deepseek V4 Flash reached 82.5% accuracy using 83,000 tokens. GPT 5.4 for at medium effort reached 82% using 3,177. Half a point of accuracy for 26 times the tokens is a subscription nobody signed up for. Inside one family, it holds two.
Claude Opus 4.7 matched 4.6's accuracy on roughly four times fewer tokens. Efficiency now moves independently of intelligence, which makes it something you shop for. Every current API hands you the dial. Claude Opus 5 shipped on July 24th with an effort setting that defaults to high so untouched you have been buying maximum effort on trivial requests since launch and reasoning tokens bill as output the expensive side of the invoice. GPT 5.6 treats effort as a ceiling rather than a floor and may spend zero reasoning tokens on a prompt it judges easy.
Verbosity tuning is knowing which of your cues deserve which setting. Cheaper than a cheap token is one you already paid for. Anthropic prices a cash read at onetenth of a normal input token. A 90% discount is a vendor telling you which part of your prompt cost them almost nothing to serve twice. One team published their numbers on July 23rd.
Same feature, same models. A badly ordered prompt captured about 18% of the available savings. After reordering, they hit an 85.8% cash hit rate and cut input costs by 77%. The changes were dull. Static instructions split from dynamic context, shared context moved ahead of conversation history, prompt blocks generated deterministically, and hit rate watched in production like any other metric.
Prompt architecture as a memory layout problem, and most teams switch caching on and rarely check whether it is landing. No cache saves you from sending a hard model an easy question. On July 30th, Open AAI cut its Luna tier by 80% to 20 cents per million input tokens while the flagship soul stayed at $5. Inside one vendor on one day, that is a 25 to1 spread. Routing decides which requests deserve the expensive end, and the research floor is old and solid.
Back in 2024, LMSYS root LLM held 95% of the Frontier models quality on MTBench while cutting cost over 85% by sending a quarter of queries to the strong model. Production teams report 40 to 70%. Routing stays available precisely because it is unglamorous. A classifier costing a fraction of a cent decides whether the next call cost a fraction of a cent or a nickel. Every skill so far is a trade and you cannot trade what you cannot measure which makes the eighth the one that makes the other seven legible.
Anthropic's January guide on agent evals is refreshingly small. Start with 20 to 50 tasks drawn from real failures, not hundreds. Measure pass at K, meaning it worked in one of K tries next to pass to the K, meaning it worked in all of them. The second number is what production depends on, and it is the one that looks worst. track turns, tokens, and latency per run, not just the verdict.
The trap is treating evals like unit tests. A unit test asserts an exact match. An eval measures semantic correctness across a distribution, so it has statistical power and 12 cases have almost none. So you have a measured, tuned, defended agent. And then the ninth skill walks in from outside software entirely, holding an invoice.
Trendforce put DRAM contract prices up 90 to 95% quarter-over-arter in the first three months of this year with NAND flash up 55 to 60. No factory burned down. Samsung SKH highinex and Micron control over 95% of DRAM and they moved wafer capacity to high bandwidth memory for AI accelerators because the margin is better. Data centers now take roughly 70% of world memory output up from 20 to 30% in 2022. SKH Highix's chief executive said in July that the shortage probably runs past 2030.
Gartner puts the combined DRAM and SSD surge near 130% across the year. 128 GB DDR5 kit has been listed above $4,000. That reprices your architecture. Context window size is a memory cost. KV cache is a memory cost.
And after the first token, generation is bandwidthbound. The model streams its entire active weight set through the processor for every token it writes. So a twochannel desktop at roughly 90 GB a second gives you 8 to 15 tokens a second on a model that would do 40 or more in VRAM. Same model, same quantization. The pipe is the ceiling and the pipe now has a price on it.
That shortage is exactly what makes the 10th skill harder and more valuable than a year ago because open weights finally caught up. Moonshot released Kimmy K3's weights on July 26th. On LLM stats terminal bench 2.1 board, it sits second at 88.3 against GPT 5.6 souls 88.8. But download is carrying a lot of weight in that sentence. K3 is 2.8 trillion parameters and about 1/2 tab of weights.
That is a rack, not a workstation. Open the way. A bridge is public. You can walk on it. You cannot take it home.
The one that lands on hardware you can actually buy is Deepseek V4 Flash. Its 3-bit build is 103 GB which fits 128 GB machine. The 8-bit build is bit identical to the official checkpoint. So local first now is narrow and expensive. Which quantization keeps the answer honest?
What the machine costs this month rather than last year? And which workloads justify owning metal instead of renting it? Open weights got good in the same year the memory to hold them doubled in price and the same buildout did both. 10 skills. If you only have time for one, the honest pick is not the fashionable one.
Learn Harness Engineering. It is the only item on this list backed by something like a controlled experiment. Same weights, same tasks, 12 points of difference on a board anyone can recheck. It also wins by containment. Context, tools, permissions, caching, and routing are all decisions made inside the harness.
Get good at the loop and you own a framework for the other five instead of five more tutorials. The counterargument deserves respect. Harnesses are commoditizing. Claude code, codeex, and cursor absorb these improvements every few weeks and most teams should not be writing their own agent runtime in 2026. That is correct, and it moves the skill rather than deleting it.
When five vendor loops differ by 12 points on your workload, measuring which one is better on your tasks is worth more than being able to write one. For whom? anyone shipping agents other people depend on the 5% this misses are solo developers on a single vendor CLI your money is in the effort dial and routing this afternoon no rewrite required the villain across all 10 is one idea that choosing the right model card is the job it is the most expensive belief in AI engineering right now and every vendor benefits from you holding it so a bet with a date on it in August 2027 the same model will still span eight points or more across harnesses on the public boards, the gap will not close because the gap is the work. Which leaves the question I cannot settle for you. If the harness really is the product, do you get better at building loops or better at judging the ones the vendors hand you?
Both are live positions right now and one of them ages