Loop vs Graph Engineering: The 48-Point Harness Secret

Two numbers published five weeks apart in 2026. Same job. They point in opposite directions. Anthropic ran the exact same prompt two ways. First, one agent alone in a loop. 20 minutes, $9. Then the same prompt through a structured harness, a planner, a builder, an evaluator. 6 hours, $200, 22 times the money. And the $9 version shipped something that looked finished and did not work. So structure wins obviously except a controlled trial published five weeks later says the exact opposite. 200 conversations per condition three domains the orchestrated graph failed 24% of them. The plain loop same model same procedure failed 11 and a half on one domain. The graph was 18 times worse. Both results are real. Both are from this year. That is the argument every serious agent team is having right now. Loop engineering or graph engineering. Let the model drive itself or wire the path yourself node by node. By the end of this, you will know which one your project is because the research settles it, just not the way either camp wanted and the number that decides it turns out to live outside the model entirely in the code you wrap around it. Start with the loop because it is the simpler object. A loop is a model, a set of tools, and a while statement. You hand it a goal. It looks at the state of the world, picks a tool, runs it, reads the result, and decides what to do next. Nobody wrote the path. The path is whatever the model decides that day. On the 7th of June, Peter Steinberger posted one sentence that collected nearly 20,000 likes. You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents. Addi Osmani, an engineering lead on Google Chrome, published the essay that named the practice the same day. And this was not just an internet thing. Boris Churnney, who runs Claude Code at Anthropic, said it on stage that month. I don't prompt Claude anymore. I have loops that are running that prompt Claude in figuring out what to do. My job is to write loops, which sounds like a movement, but it started as a joke. Back in July 2025, Jeffrey Huntley wrote a bash while loop that fed a coding agent the same prompt against the same spec over and over, a fresh instance every round. He called it Ralph after Ralph Wigum. It was not supposed to work. It works well enough that OpenAI's own engineering blog cites it by name. In February, OpenAI described shipping a real product with zero lines of manually written code about a million lines, 1500 pull requests, three engineers, roughly a tenth of the time. And their own word for that architecture is a Ralph Wiggum loop. The details matter more than the headline. Single codeex runs regularly work one task for over 6 hours often while the humans are asleep and the instruction file is deliberately about a 100 lines. Give codeex a map they wrote not a thousandpage instruction manual. There was a bill attached to that. The team used to spend every Friday 20% of the working week cleaning up what they called AI slop before they replace that job with recurring garbage collection agents. The loop bought speed and charged rent. The purest version of the idea is about a 100 lines of Python. Minisw SWE agent has no tool calling API, no special scaffold, one tool, bash, and a completely linear history. It scores over 74% on S.Bench verified. And here's the part that gets skipped. SWEBench verified, the leaderboard this industry quotes at each other, runs every single model through that same 100line loop. The maintainers describe it in their own words. No tools, no special scaffold structure, just a simple React agent loop. The benchmark of record is a wild statement. At the top of that board, Claude Opus at 76.8% for 75 cents a task. Further down the same board, Miniax M2 5 at 75.8 for 7. One point of quality about 11 times the price. Identical loop underneath. Now the trial from the cold open. In April, a team put an entire customer service procedure into the system prompt and let one model self-chestrate. Then they built the same procedure as a Lang graph orchestrator. 14 nodes in two domains, 55 in the third and pushed 200 conversations through each. Travel booking, the graph failed 24%. The prompt only loop 11 and a half. Insurance 17 versus 5. Support scheduling 9% versus half a percent. The paper's title is not subtle in context prompting obsoletes agent orchestration for procedural tasks. So the loop is cheaper, simpler, sits under the leaderboards and just beat a handbuilt graph on the graph's home turf. Which raises the obvious question, if it is that good, why is anyone still drawing boxes and arrows? Because of one line of arithmetic. Reliability compounds. A 10-step process where each step works 90% of the time completes about 35% of the time. At 95% per step, you get 59. At 85, you get 20. That is P to the N. It is not a bug in anyone's code. It is what multiplication does to a chain. And a free running loop does not stop at 10 steps. Arena measured a full week of real agent traffic. 160,000 tasks, just over 2 million tool calls. The average session made about 16, but 17% of sessions made 26 or more. The 99th percentile sits near 200. and the longest runs went past 2,000. The heaviest single sessions in that window are worth saying out loud. Claude Opus, 140 turns, 448 tool calls. GPT 5.5, 84 turns, 676 calls. Multiply 90% by itself 600 times and tell me how that ends. And the real decay is worse than the arithmetic predicts. One study this year ran the geometric model against measured long horizon performance. The model predicted 33%. What actually happened was 22. The gap is the interesting part. Errors inside a loop are not independent. Once the model commits to a wrong hypothesis, it keeps building on top of it. In that same study, episodes that died before finishing even the first subtask went from 1% on short tasks to 25% on the long ones. Work it backwards and the ceiling gets brutal. To run a thousand dependent steps with better than even odds, one survey puts the requirement at per step reliability above 99.93%. Fewer than seven mistakes in 10,000. No agent is close. Then there is what happens to the context itself. Open AI publishes this for its own model. On a multi- needle retrieval test, accuracy falls from 97% at 4,000 tokens to 37% at half a million. Graph traversal falls from 93 to 21. and loops get there. In Arena's week, 32% of sessions ended with more than 128,000 input tokens in the final turn. 22% past a quarter of a million, 8% over a million. It also fails in the dumbest available way. A July paper scanned 6 12,000 agent repositories and confirmed 68 real infinite loop bugs across 47 projects, unbounded feedback paths that turn one request into cost exhaustion, and repeated side effects out in the world. There is a worst case for that and it is not hypothetical. One enterprise ran up $500 million of model spend in 30 days after failing to set any cap at all and the code that comes out degrades as it goes. Slop Codebench ran 15 coding agents across 36 multi-stage problems. Not one agent solved a single problem end to end. The best passed 14.8% of checkpoints structure eroded in 77% of trajectories and against 473 real repositories. The agent code came out twice as verbose. So you add structure. That is the graph. Instead of one loop deciding everything, you declare the states, the transitions, and the checkpoints. Nodes do work. Edges say what is allowed to follow what. The model still thinks, but it thinks inside a node. And the path between nodes is code you wrote and can read. That is the direct answer to compounding error. Every checkpoint is a place a run can be inspected, retried, or resumed instead of drifting. You are not making the model more reliable. You are shortening how far one mistake gets to travel and it holds up in production. The paper that states the 35% math also reports its own deployment 8,728 workflow runs across three clinical sites over a year, 97% completion with the failures landing on external integrations rather than on the graph. Determinism is also just cheaper. A March paper replaced the reasoning based router with a cost-weed tool graph and Dixstra's algorithm same correctness as a react loop with 93% fewer control plane model calls 9 instead of 123. Compiling the procedure helps too. Covenant turns a natural language workflow into an explicit control flow graph. Success on its benchmark went from 50% to 83 and workflow misalignment failures dropped from 42 1.5% to under 16. Graphbit runs the whole graph in a rust engine and reports 67.6% on Gaia with zero framework induced hallucinations and about 12 milliseconds of overhead. When the framework cannot hallucinate a route, it does not. The most used graph runtime is Langraph. Over 65 million downloads a month, 3 years old, shipping a release roughly every 2 weeks. The one before last added durable error handler resom across a host crash which tells you exactly who is buying because underneath a lot of production graphs sits durable execution. Temporal raised $300 million at a $5 billion valuation in February on the back of 9.1 trillion lifetime action executions. 1.86 trillion of those came from AI native companies. Revenue up more than 380% year-over-year. installs up 500% past 20 million a month. Whatever the internet thinks of graphs, the market for replayable crash proof execution is accelerating. So structure has receipts, revenue, and the arithmetic on its side, which makes the next fact genuinely hard to explain. The company that sells the graph took the graph out of its own flagship product. In July, in a post celebrating 3 years of Langraph, Harrison Chase and Sydney Wrl wrote this about their own deep research agent. We built early deep research on predefined langraph workflows then moved to a more agentic core loop. They were not alone. GPT researcher 28,000 stars pulled out its Langraph multi-agent pipeline and rebuilt it on a harness. Their readme says the same pipeline can be achieved with far less code because planning, delegation, and context management are emergent from the agent harness rather than a hard-coded graph. Their migration table is the whole story in miniature. An editor node plus a research state object became one built-in to-do tool. A nested subgraph per section became a task tool that spawned sub aents. State keys passed between nodes became plain files on disk. Three graph constructs deleted, replaced by things the model already knew how to use. And Langchain's own loop-shaped product, deep agents, took 27,000 stars in about 12 months. Langraph has 38,000 after 3 years of being the default answer. Research points the same way. A study across 208 production-drived enterprise scenarios found the graph approach wins at small scale, but its overhead gets worse as the scale grows, while the plain loop stays more robust because it handles failures incrementally instead of at a checkpoint. With one detail I did not expect. At 200 agents, scale rather than task complexity is what dominates and the simple tasks degrade more sharply than the complex ones. The structure you added for the hard cases is taxing the easy ones. The deepest version of the argument is a single sentence anthropic published in April. Harnesses encode assumptions that go stale as models improve. They hyperlink the words go stale straight to Rich Sutton's bitter lesson, which is about as pointed as an engineering blog gets. And they showed their working. Sonnet 45 had what they called context anxiety. So they engineered context resets into the harness. On Opus 45, that behavior was simply gone. Their words, the resets had become dead weight. The same thing happened to the expensive harness from the cold open. On the next model, the builder ran coherently for over two hours with no sprint decomposition at all, and the $200 run came down to $124.70. Every node you add is a bet against the next model release, which is where the real answer starts, and it is not loop or graph. Run the same benchmark twice. Once with the scaffold fixed and the model changing, once with the model fixed and the scaffold changing. Then look at which number actually moves under one standardized scaffold. The six leading frontier models span 4.9 points. 4.9 that is the entire spread of the frontier measured on equal footing on a hard software benchmark. Now hold the model still and change the scaffold. Clawed sonnet 45 goes from 68% down to 34. 34 points identical weights. GPT5 medium 46 down to 12. One smaller model swings almost 48 points on scaffold alone. So the model you pick is worth about five points and the code you wrap around it is worth up to 48 which makes the scaffold the largest single lever in the system. Most teams still treat it as plumbing. One clean example on SWE Pro. Claude Opus 45 scores 45.9% inside the standardized scaffold and 55.4 inside Claude code. Same model 9 and a half points purely from the harness around it. Lang Chain proved it deliberately. They froze the model, changed only the harness, and moved their coding agent from 52.8% to 66.5 on Terminal Bench, 30thish on the leaderboard to top five with the weights untouched. One finding inside that run is worth stealing. Turning reasoning all the way up made things worse. Everything at extra high scored 53.9% because the agents timed out. At plain high, 63.6. More thinking, fewer finished tasks. Stanford went further and had an agent rewrite the harness code itself. Their system beat Claude code 76.4% to 58 on terminal bench 2, an 18point gap, same model produced entirely by the code around it. The mechanism is almost silly. It feeds its optimizer up to 10 million tokens of raw execution logs per step. Every prior method they surveyed, self-refine, OPRO, Techgrad, GPO, capped out around 26,000. It wins by reading everything that happened. So what happened to the argument? In six weeks, this industry renamed its own job twice and then the graph vendor dissolved the whole framing. In that anniversary post, Lchain wrote two sentences that end the debate. Agent graphs are usually not DAGs because production agents need cycles. And then loops are simple graphs. Loop engineering is not an alternative to graphs so much as a simple version of them. David Kursid who built Xate had already put it in eight words. A loop is just a directed cyclic graph which means the question was never loop versus graph. It is how much of the path you are willing to write down and every answer sits on one line from a while statement at one end to 55 declared nodes at the other. Anthropic gave the clearest rule for where to sit on that line in the same post as the $200 bill. The structure is worth its cost when the task sits beyond what the current model does reliably on its own, not before that and not forever. So the practical version, if your task fits inside one context window and you can cheaply check the answer, run a loop and spend what you saved on checking it. If it runs for hours, touches money or medicine, or has to survive a machine dying halfway through, declare the states. That is what 8,000 clinical runs at 97% bought and write your structure down expecting to delete it. Anthropic deleted their context resets then deleted their sprint decomposition. Every scaffold has a half-life and right now that half-life is roughly one model release. The thing to take from this year is that the argument moved. The useful question stopped being which framework you picked. It became how much freedom your task can survive. And that number has gone up every few months in public with receipts.