Agentic AI: Should the orchestra play itself?

Written By Sébastien Simpson

If the model writes the work and grades the work, who is left to say it's wrong?

Sakana AI's new paper on recursive self-improvement hides its best detail in the methods. The researchers let one model play every role in the loop, including judging its own output. Then they asked two outside models, GPT-5.5 and Claude Opus 4.8, to score the results, and kept those scores hidden from the loop. Without them, the system had no way to tell whether it was improving or only growing more sure of itself.


After two rounds, a 27B open-weights model scored 1.2 to 1.6 times its starting level per output token on four research benchmarks. On Terminal-Bench 2.0 and SWE-bench Verified it landed slightly below where it started, at 0.96 and 0.94. The training set was twelve synthetic tasks, so this is a proof of concept. The conclusion is the part worth carrying out of the lab: when one model is both optimizer and evaluator, a shared blind spot can produce a flawed result, get accepted as an improvement, and feed the next version. Any program lead has seen a smaller version of this, a team reviewing its own work against its own standards and then shipping it.

Nobody has closed the loop.

RAND's assessment, published October 12, says several frontier labs report that most new code is now AI-generated, but no disclosed system sets its own research agenda or designs its successors. There is also no systematic reporting on how much AI research is automated, so the real pace is unknown. What exists is a set of partial loops, and a person sits in each one.

Enterprises seem to be reaching the same conclusion from the other direction. In VentureBeat's August survey, the share of organizations that let agents push production changes on automated evaluation alone, or are building toward it, fell from 75% in July to 56%. The sample is 140 respondents and the waves are a month apart, so read it as a direction. Of the 38 whose organizations already allow changes without human approval, only 10 named automated quality checks on live outputs as their main way of monitoring.


Your teams run partial loops too. An agent drafts the weekly project update, a person corrects it, and the corrections shape next week's prompt. The evaluator is a human outside the model. That is the advantage.

Faster output, same delivery date.

GitLab, a public company with roughly $1 billion in annual revenue and a rival to Microsoft's GitHub, makes a platform many software teams use to plan, review, test and ship code, which gives it a view of where AI-written work gets stuck. Its data, reported by CIO&Leader on October 6, shows that 78% of organizations see faster code commits while 21% report productivity gains across the full lifecycle. GitLab sells the platform it is describing, so weigh it accordingly. The mechanism it names will be familiar to anyone who has run a program: every AI-written change still needs review, testing, security checks and approval, and manual handoffs turn the time saved into queues. In GitLab's account, writing code was already the smallest part of delivery, so speeding it up moves the constraint without moving the finish date.


The pattern isn't unique to engineering. Briefs, creative reviews and client approvals queue the same way when production speeds up and the approval path stays as it was. A concept round that takes a day instead of three still waits for the same Thursday review.


A schedule is a chain of handoffs, and its date is set by the slowest link. Shorten the links you can see and the date holds. Program leads end up explaining why a team that is visibly producing more is delivering on the same day, and the honest answer is that nobody changed the part of the plan that sets the date.


That is why the unit of measure matters. GitLab's proposed fix is a metric: cost per accepted change, meaning the total cost of people, AI and tooling divided by the number of changes that reach production. Retries and abandoned work add cost without adding to the count, so waste shows up in the number. Outside software, I'd translate it as cost per approved deliverable: the hours and tool spend behind each piece the client accepted, with rework included.

Outputs are becoming outcomes.

Once production is cheap, the output stops being the thing anyone wants to pay for, and the advertising industry is working through this in public. Digiday reported in July that the CEO of Cheil Agency Network is moving all of its agencies from time-and-materials to output-based pricing, with all new business on those terms and half of existing clients already converted. The same piece names the obstacle. Nobody has agreed on how to define and measure what the AI delivered, so agencies are pricing results they can't fully account for, and procurement teams that expected AI to shrink the bill read a fee tied to AI-assisted revenue as the opposite of what they were promised. Digiday's October agency survey, which covered only 30 professionals, shows the bill arriving: 31% said AI cost more than expected to deploy, and 59% are selling AI as a new product instead of recovering it through hours.


Adweek's recent reporting shows the same tension. It reported that Horizon Media's president called headcount-based billing "the old system" and expects brands to stop paying for it within five years. A Horizon spokesperson said 25% of its clients have a performance-based component today. Adweek's follow-up adds the catch: in an outcome-based deal the agency carries most of the risk, because missing the agreed benchmark means not getting paid, and brands have been slow to move. Digiday is programming a session at its Media Buying Summit later this month on making outcomes-based relationships work, and its description concedes the idea is attractive in theory and much harder in practice.


Outcome contracts pull the checkpoint to the start of a project. Someone has to write down, before the work begins, what result it is meant to produce, how it will be measured, who verifies it, and what happens when the data is contested. That is the contract idea from the Sakana paper, applied to a client engagement: define done, and define the evidence.


It also changes what a program lead reports. Items closed, deliverables shipped and velocity describe activity. An outcome needs a baseline and time to show up, so a program can ship every output on schedule and still not know whether it worked. I'd add one outcome measure to every status report, with a named owner for the data behind it.

Agent sprawl is an ownership problem.

IBM's guidance on coordinating agents, published October 12, describes how adoption starts as separate initiatives: one team deploys an employee assistant, another automates a workflow on a different platform. As they multiply, it gets harder to see which agents are running, who owns them and what they can access. IBM's answer starts with accountability. Assign a business owner for the outcome and an operational owner for each agent, and decide before the agent sits in a critical workflow who can restrict its access, pause it or restore an earlier version. IBM sells an orchestration product, and the advice reads like what a product team would write. It is still the clearest statement of the owner split I've seen.

The agency version of the problem is less formal. A technology chief at media agency Novus told Digiday that every team is building its own tool, and that someone central will have to unify them and work out which are real.

GitLab describes the fragmentation from the tooling side. A team that plans in Jira, reviews in GitHub, builds in Jenkins, scans with Snyk, stores in JFrog and deploys with Harness has six identity models and six logs, so no single record shows the whole change. It calls the result a shadow software factory: each developer orchestrating agents under personal rules, with local credentials. Its countermeasures are worth borrowing. An agent runs under the permissions of the person who invoked it and never more. Low-risk changes can pass agentic review once required checks pass, higher-risk changes escalate to a human, and both leave a record in the same system.

Both answers reduce to one test. When an agent does something wrong, who can stop it, and what can that person see? Most organizations can't answer it for the agents they already run.

One agency is running the experiment on itself.

Josh Lowman runs Gold Front, a San Francisco category-design studio. Josh is amazingly talented - and his work is sharp and crisp. He has spent the year building an AI operating system for his own agency. In a March episode of his podcast, he described Gold Front OS, an agent setup on Claude Code with a memory layer a colleague built. He says it changed how he runs the whole agency and already knows his to-do list before he does. The notes for an August conversation on a storytelling podcast list agents built around Gold Front's strategy frameworks, a chapter on the danger of overbuilding with AI, and one on why processed creativity quickly loses its value.


The order of operations is what I find useful. The memory and the frameworks come first, so every agent starts from the studio's own context and judgment. I couldn't find published figures for hours saved or turnaround, so these are the questions I'd ask him. How long does a first-draft positioning deliverable take now compared with a year ago? Which steps does the system never touch? Who reviews what it produces? Gold Front's studio has a small and nimble team, which means the operating system likely answers to one identified owner. A larger organization has to decide who the owner is as well, and give that person the hours.

The operating system is mostly contracts.

An AI operating system, as I'm using the term, is the layer that holds a firm's context, rules and approved actions so that agents and people work from the same source. Orchestration is what it does with them: it decides what runs, in what order, with what context, and what has to come back before the next step. IBM calls its version a control plane and GitLab calls its version a context layer. Josh Lowman calls his Gold Front OS. I made the case in March that orchestration is program management with a different instrument panel, and the vendors have since built products around the same idea.

Sakana's paper adds a detail about what that layer should contain. The researchers looked at what their evolved team workflows ended up encoding. Over successive iterations, roles and instructions carried less task-specific information. A role like "careful data scientist" fits any task. Contracts and call order carried more. A contract is what each agent must hand back, including required checks. In the paper's finance example, the data specialist has to report whether the training, validation and test periods are in the right order, so a backtest can't quietly use information from the future. The sample was eight or nine tasks, and the authors flag that, but the direction is hard to ignore. A system optimizing itself converged on what good program managers insist on early: define done, define the sequence, and make each handoff prove something.

The paper illustrates its approach with a 1600 engraving of a printing workshop. Typesetters work on the left, the press runs on the right, and in the foreground, sheets are laid out for proofreading. The press multiplied the pages, and proofreading kept its own station in the room.

What the customer meets.

Customers are sending agents of their own. On October 10 I wrote about Meta's shopping agent showing up at a merchant without saying who it was, and Amazon blocking it. If an assistant can browse your storefront and answer questions about your brand somewhere else, the shared context inside your operating system has to hold up outside it. The checkpoint applies to what leaves the building as well as to what moves between teams.

Four levels of delegation.

I sort work by how much judgment it needs, and the checkpoint moves with it.


The first level is execution: status narratives, meeting recaps, format and naming checks. The agent drafts and I finish, checking every figure against its source before anything goes out. The second is insight: comparing a plan with a tracker, spotting a date that doesn't match or a dependency nobody owns. The agent flags and I investigate, and every flag stays a hypothesis until I've confirmed it, since a flag built on untidy data is often an artifact. The third is strategy: options and knock-on effects for a scope or timeline change, workshop notes reconciled before a planning session, a view of who owns what when two documents disagree. The agent holds the context and I decide. It knows only what it has been given and can state wrong things with confidence, so its work gets the scrutiny I'd give a sharp new hire's first memo.

The fourth level is the one this piece opened with: systems that improve their own workflows and, eventually, their own research. Nobody should run that on trust alone. The checkpoint moves outside the system, to independent judges like the two Sakana kept hidden, to the reporting RAND says doesn't exist yet, and to institutions that can absorb the pace. The human job doesn't shrink as the levels rise. It moves from editor to investigator to decision-maker to outside judge.

What makes this orchestration and not a pile of prompts is the plumbing. Drafts land in the tracker the team already uses and never in a parallel one. Flags go to a queue with a named owner. Briefs cite the documents they came from. Nothing writes to a shared system until a person approves it, and every output traces back to a source. I haven't measured results at each level, so treat the ladder as a way to sort work, not a benchmark. I delegate upward only after the level below has held up under review.

What I'd put in next quarter's plan.

Pick one workflow with a clear owner and a measurable outcome, and choose it for its approvals rather than its technology. IBM's advice matches mine here. Connect that workflow end to end before adding a second.


Name two owners for every agent: a business owner for the outcome and an operational owner who can pause it. Write down what the agent may touch, and keep it within the permissions of the person who started it. In September I wrote about the checkpoint nobody owns between "AI produced this" and "this shipped". It still needs a name on it.


Route review by risk. Low-risk output passes defined checks and gets logged. High-risk output goes to a named person with time to read it. For each type of output, write down who reviews it, against which source, and what gets logged.


Write the outcome and its evidence into the next scope of work before the work starts: the result, the measure, the verifier and the baseline.


Measure a baseline for a month, then set targets. Track time from request to accepted result, rework rounds per deliverable, and how often a reviewer overrides the agent, alongside one outcome the work is meant to move. Put review hours on the schedule. If the plan assumes AI output arrives already reviewed, the plan is wrong.


Sakana's workflow search is noisy, by the authors' own account: strong iterations followed by sharp regressions. A human review standard can do the same. The one that was tight in March can loosen by October because everyone is busy and the drafts have been good lately. I haven't found a published measure of how fast review standards drift. If you've seen one, send it to me.

Further Reading.


Sébastien Simpson
Next

Agents forgot to introduce themselves.