Harness engineering: the system around the model

An agent changes the code, runs a command and announces that the job is done. The patch looks plausible. The summary is convincing. Then someone opens the application and finds that the feature never worked.

That gap between a convincing account and a verified result is where harness engineering begins. A stronger model may make fewer mistakes. A longer prompt may remind it to check its work. Neither, by itself, gives the surrounding software a dependable way to establish what happened.

The missing piece is the system around the model: the layer that manages context, controls actions, preserves state and asks for evidence before accepting completion.

The model is one part of the system

A model can reason about a task and propose the next action. The harness determines what it can observe, which tools it can call, what those calls are allowed to change and how their results become the next step’s context. The environment is where those actions meet reality: a repository, a database, a browser, a service or a person.

This is the framing in Hailin Zhong and Shengxin Zhu’s May 2026 preprint, AI Harness Engineering. Their proposed H0–H3 ladder adds tools, persistent context and verification in stages. The useful distinction is the evidence each stage leaves behind.

The paper includes a controlled validation task; it does not establish a general performance law. I read it as a useful architectural proposal. It makes the unit of analysis the complete system, and makes the evidence of a run as important as its final patch.

Give the harness a clear job

In practice, I group the work into seven responsibilities. The boundaries matter more than the terminology: each responsibility needs an owner outside the model’s next response.

Contract

Define the task, the evidence that counts as done and the conditions that end a run.

Context

Select the relevant sources, instructions and prior decisions for this step.

State and memory

Distinguish what was proposed, what was attempted and what actually completed.

Tools

Expose explicit operations with validated inputs and useful, bounded results.

Authority

Enforce access rules and require approval before an action crosses its permitted scope.

Verification

Check claims against tests, sources, transaction records or the running product.

Recovery

Record failures, bound retries and provide a route to repair or human handover.

Some of this is ordinary software engineering: schemas, state machines, access controls and logs. The difficulty is putting those pieces around a component whose next action is not fully predictable. Good architecture makes that uncertainty manageable without pretending it has disappeared.

Put the rule where the action happens

Consider a refund that requires approval. Writing “ask a person first” in the prompt expresses the rule. Checking an approval record at the execution boundary enforces it.

The distinction is concrete. The tool should validate that approval belongs to this action, this amount and this recipient before it sends anything to the payment service. The model can propose a refund; it cannot manufacture the authority to issue one.

Joongho Ahn and Moonsoo Kim examine this move in their July 2026 preprint, From Prompts to Contracts. They place source boundaries, routing rules, trace requirements and output checks in code around a replaceable model. Their code-owned checks held across 270 composition-boundary runs with three hosted models; deliberate faults activated the validators.

The scope matters: fixed scenarios over public data from five Korean corporate groups. Those results support the pattern within that evaluation. They do not mean all agent behaviour can be made correct by adding validators. A check can enforce only the property it was designed to inspect.

Build a loop that can accept a refusal

Prompts still matter. Clear tool descriptions reduce ambiguity; selected context gives the model a better basis for its next decision. Anthropic’s Writing effective tools for agents makes this practical through tool naming, descriptions and response design.

But useful instructions sit inside a larger loop. After the model proposes an action, something must establish whether it may run and whether it worked.

The check should reach beyond the model’s account of its own work. A coding agent needs tests against the change. A deployment needs a health check on the running service. A database action needs a transaction result. Repeating “review your answer” does not create any of those signals.

Anthropic’s Building effective agents describes this as obtaining ground truth from the environment. Its later work on long-running agents makes the approach tangible: persistent progress records, a feature list and browser tests that exercise the actual application. These are ways to make the next step depend on observable state.

Read the trace, then check the outcome

A successful final answer can conceal a poor execution path: unnecessary retries, a wrong tool call, an approval requested too late. A trace makes that path inspectable. Outcome checks establish what the path actually achieved.

Anthropic’s Demystifying evals for AI agents treats evaluation as a test of the model, harness, tools and environment together. One practical lesson is to grade the state left behind: whether the file changed, the tests passed or the expected record exists. The transcript helps explain how the system got there.

That makes traces part of the product’s operating record. They should capture tool calls, policy decisions, approvals, results and enough context to investigate a failure, with appropriate handling of sensitive data. A polished completion message is useful to the reader; it is not a substitute for that record.

Where this meets my own work

I came to this problem through autonomy and the handover to a person. A machine recommends an action, but the evidence behind it is often lost before the person has to decide.

NODERIQ is concerned with carrying that evidence to the decision. NowFlow makes approval an explicit routing step. QFlow Studio keeps the brief, generated source, provider route and run evidence together so an experiment can be reconstructed. In the agentic software-testing work I lead, evaluation and oversight belong before the release decision.

These systems work in different domains, but they meet the same architectural question: where does a proposal acquire authority, and what evidence survives the action?

Design for the moment something goes wrong

The most revealing review of an agent system starts with a failed run. Can we tell what it saw? Can we reconstruct the action it attempted? Did a policy stop it in time? Does the next attempt know what already happened?

Those questions turn harness engineering into concrete design work. Define completion before execution. Put enforceable rules at the action boundary. Preserve enough state to recover. Check the result against the world.

A capable model creates possibilities. The surrounding system determines which possibilities become dependable work.