The Harness Is Everything You Can Swap Without Retraining the Model.

The problem

If you debug agents for a living, you have had this argument. Someone says the model is bad at long tasks. Someone else says the scaffold is wrong. A third says it is tooling. Nobody settles it, because nobody agrees where the model stops and the system around it starts.

The word for that system is harness. It currently means two different things.

Anthropic, in January 2026, defined an agent harness, or scaffold, as "the system that enables a model to act as an agent: it processes inputs, orchestrates tool calls, and returns results." That is a runtime. It sits between the caller and the weights. Its job is the loop.

OpenAI's harness engineering post, published February 11, 2026, contains almost none of that. It is about a docs/ directory treated as the system of record. A roughly 100-line AGENTS.md used as a table of contents rather than an encyclopedia. Linters that enforce dependency direction between layers. Per-worktree app instances, with an observability stack the agent queries in LogQL and PromQL. Their harness is the repository and the environment around it.

Both are first-party. Both are right about their own system. They are not describing the same object. That is the tell: if two of the best-resourced agent teams alive use one word for two things, your own boundary is probably wrong too.

A definition you can test

I use a behavioral definition rather than a structural one.

The agent harness is everything that determines what the model sees, what it is permitted to do, and when it stops: every component you could change to alter the outcome while the weights stay fixed.

The operational form is a substitution test. Hold the weights constant. Change one thing. See whether the result moves.

That settles the disagreement rather than papering over it. OpenAI's repository is harness here, because making it legible changed what Codex produced from the same model. Anthropic's loop is harness for the same reason. Both worked one surface at different points.

What a harness is not

Not the model. The weights are what you hold fixed. Not the product: billing and accounts change no agent outcome. And not a synonym for "orchestrator," which answers sequencing and stays silent on context, policy, and stop conditions.

Where the boundary falls

Eight layers, one boundary. The stack below runs in the same order as the legend above — trace a color from the hero animation straight down into this list.

Two of those rows get collapsed constantly. Anthropic's line on workflows is clean: they are "orchestrated through predefined code paths," while agents "dynamically direct their own processes and tool usage." A workflow is something a harness executes, not a rival to it. "Runtime" is worse. It gets used for the process executing tool calls and, loosely, for the whole harness. Say which you mean.

None of this implies a minimum size. A short loop that exposes two tools, gates one permission, and records state is still a harness; the stack above lists responsibilities, not a parts count. Complexity earns its place only when a real failure mode requires it — the same argument the depreciation section below makes about pruning, not just extending.

MCP is a protocol, not a harness

The specification is explicit about its scope: base protocol, lifecycle, authorization, server and client features. No agent loop. No context budget. No stop condition. More telling still, the server-primitives page splits control three ways: prompts user-controlled, resources application-controlled, tools model-controlled. The tools page then warns that "there SHOULD always be a human in the loop with the ability to deny tool invocations." MCP delegates policy to the host, and that host is the harness.

The loop, as code

The diagram above compresses to about nine lines. Only one of them touches the model.

# everything here is harness — except the one call to model.generate()
while not done:
    context  = assemble_context(state, budget)        # 2 · context
    response = model.generate(context, tools=exposed) # 3 · the model — held fixed
    action   = parse(response)                         # 4 · parse

    if not policy.allows(action):                    # 5 · policy gate
        action = policy.escalate(action)               #   allow / ask / deny

    result = dispatch(action)                          # 6 · dispatch
    state  = render(state, result)                     # 7 · rendered as data, not instructions

    if stop.reached(state, budget):                   # 8 · stop test
        done = verifier.check(state)                   # 9 · probes real state, not the model's claim

The model produces response. Every other line — assemble_context, policy, dispatch, render, stop, verifier — is a component you can swap without retraining anything, and the substitution test from above applies to each one individually: change how policy.allows() decides, or what verifier.check() actually probes, and the agent's behavior moves while model.generate() stays byte-identical. That is the whole definition, in a form you can set a breakpoint in.

A benchmark number is a joint measurement

Anthropic reported 49% on SWE-bench Verified for the upgraded Claude 3.5 Sonnet, against a 45% previous state of the art. The same post described the scaffold: a prompt, a Bash tool, an Edit tool, deliberately minimal, "to give as much control as possible to the language model itself." Then came the sentence that matters:

The performance of an agent on SWE-bench can vary significantly based on this scaffolding, even when using the same underlying AI model.Anthropic, Raising the bar on SWE-bench Verified

A vendor conceding its own headline number is scaffold-sensitive is unusually clean evidence.

SWE-agent made the same point eight months earlier, from the other direction. It introduced the agent-computer interface and reported 12.5% pass@1, "far exceeding the previous state-of-the-art achieved with non-interactive LMs." The variable was the interface, not the model.

In SWE-bench's own repo, harness means the grader

Sharper evidence sits in the benchmark's own source tree. Look at what the directory named harness contains — and notice which color it belongs to on the legend: this is the evaluation harness, not the agent harness, even though it shares the name.

# the grader — evaluation harness, colored like the diagram above
harness/
├── docker_build.py
├── test_spec/
├── log_parsers/
├── grading.py        # reasons about FAIL_TO_PASS / PASS_TO_PASS
└── reporting.py

# the model-calling code lives one level away — agent harness territory
inference/          # sibling directory, not inside harness/

The grading module reasons about FAIL_TO_PASS and PASS_TO_PASS statuses. No agent loop appears in it. The model-calling code lives in a sibling directory, swebench/inference/ — a split visible directly in the source, and confirmed by the original SWE-bench paper, which describes the benchmark's 2,294 task instances as a grading harness built around real GitHub issues and their test suites, independent of whichever agent later attempts them.

So in the most-cited agent benchmark, harness already means the grader. Three systems produce any agent number: the model, the harness that drove it, the evaluation harness that graded it. Crediting the first alone is a category error.

Verification is not part of the definition

A common formulation has the harness returning a verified result. That does not survive the sources. Anthropic's own definition says "returns results." No verification in it. And in the November 2025 write-up on long-running agents, verification is something the team had to build: the coding agent was required to run tests before marking a feature complete, because otherwise it did not.

If verification is definitional, the many production loops returning unverified output are not harnesses. That is false, and it hides the gap you most want to find. Verification is step 9 and a maturity property, not a membership test. Calling it optional is what lets you notice it missing.

Localizing a failure

Every symptom below has at least two plausible causes on opposite sides of the boundary — the tag column marks which layer, colored to match the legend.

Row two is the one most tables omit. Prompt injection looks like a model failure, but the mechanism is a harness one: untrusted bytes reached the instruction channel because the harness rendered tool output as trusted context rather than as data. That is a claim about mechanism, not frequency — I have no published rate for how often injection incidents trace to that specific rendering choice — but it is the assumption baked into MCP's own tools spec, which insists on a human able to deny invocations: the protocol expects the host, not the model, to enforce policy.

One failure, five layers

The agent reports a migration applied; the column is not there. Walk the boundary outward from the model, cheapest check first:

1. EvaluatorGraded the final message rather than the schema — the cheapest thing to rule out first.
2. EnvironmentThe container may have torn down before the write flushed.
3. ToolThe tool may have returned exit 0 on a partial write.
4. HarnessNo post-condition probe existed, so it believed the model's claim.
5. ModelMay have planned correctly and claimed falsely — the most expensive layer to confirm.

One symptom, five candidates. Only a state probe plus the trace separates them.

One task, five failures

The walkthrough above manufactures one symptom and asks which of five layers caused it. Run it the other way: one task, five separate attempts, a different failure independently injected at each layer. Same request every time — fix the failing authentication tests and open a pull request — so what changes is only where the loop breaks, not what it was asked to do.

# same task, run five times, one layer broken per run
Fix the failing authentication tests and open a pull request.

# the correct run maps straight onto the pseudocode above:
assemble_context()  → repo state, failing test's stack trace, prior commits
model.generate()    → proposes a patch to the clock-skew check
dispatch()          → writes the patch, reruns the suite
render()             → suite output fed back as data
stop / verifier      → tests pass, PR opened against the real diff

Five failures, one visible symptom each time: "the task didn't work." The five-layer table earlier in this piece exists because that symptom alone never tells you which of these five you're looking at — and the confirmation step is different in every case, which is the actual argument for treating the layers as distinct in the first place.

A harness is a depreciating asset

Every component encodes an assumption about what the model cannot do alone. Anthropic's March 2026 harness-design write-up says so: "every component in a harness encodes an assumption about what the model can't do on its own, and those assumptions are worth stress testing." The method was ablation: "removing one component at a time and reviewing what impact it had on the final result." A sprint-decomposition construct was deleted outright: a stronger model could decompose unaided.

Both extremes are wrong, and the same vendor refutes each: its number moves with scaffolding at fixed weights, and deleting components improved the result. Neither publishes a clean scaffold-only delta, so I will not quote one.

Harness components are hypotheses with an expiry date.

Running your own ablation

Neither vendor publishes a clean scaffold-only delta, so treat their write-ups as evidence that the method works, not a number to cite. The method itself is cheap enough to run on your own system:

Hold constant: model version, task set, repository state, evaluator. Change exactly one harness component per run — system prompt, tool surface, retry policy, context-assembly strategy, verification step, budget. Everything else stays pinned, including the grader.

Per run, record what the checklist above already asks you to track: success rate, tool calls per task, retries before success or abandonment, and which of the five layers absorbed each failure. A component earns its place if removing it drops success rate or pushes failures onto a layer with no recovery path; it is a candidate for deletion if the number does not move. Run it again after your next model upgrade — the March 2026 write-up's sprint-decomposition deletion is exactly this test, repeated, catching a component that used to matter and stopped.

In my own setup they are inspectable before they run rather than reconstructed after the fact from a transcript. The manifest is declarative: each entry binds a trigger, a model route, source-trust rules, side-effect classes, and closeout verifiers. One entry treats an unauthenticated social-media post as untrusted by policy — it checks the post's own linked sources before using anything from it, and blocks every social-platform side effect regardless of what the model concludes. The point isn't the specific rule; it's that the rule is readable in one place before the agent runs, not inferred afterward from what it happened to do.

What to ask

Tick the ones you can currently answer about your own system.

Limitations

Source register