Harnesses and long-running agents

An agent harness is the system around model inference: prompt assembly, tools, permissions, execution, state, events, approvals, compaction, and the client experience. Reliable agents come from improving this whole loop, not only from changing the prompt.

Model the loop explicitly

A turn can contain many cycles:

1. assemble instructions, environment state, conversation items, and available tools;
2. ask the model for the next action;
3. execute a tool call within its own permission boundary;
4. append the call and result to state;
5. repeat until the model emits a final response or pauses for input.

The primary output may be a code or filesystem change, not the final message. Persist tool calls, artifacts, diffs, validation, and approvals as first-class events so the outcome can be reconstructed and audited.

Each tool is responsible for its own security boundary. A sandbox around the default shell does not automatically constrain external or MCP-provided tools. Treat tool schemas, permissions, authentication, and output trust independently.

Separate runtime from clients

For a reusable harness, keep agent state and execution in a long-lived runtime and expose a stable, bidirectional event protocol to clients. Model at least:

• a thread, the durable conversation and configuration boundary;
• a turn, one user request through completion, cancellation, or pause;
• an item, a message, tool call, result, reasoning summary, approval, diff, or other increment within a turn.

One client request can produce many streamed events, and the runtime may need to request approval from the client mid-turn. The server, not an ephemeral UI, should be the source of truth for long-running work so another client can reconnect and catch up.

OpenAI’s Codex App Server is a concrete implementation using a long-lived process and bidirectional JSON-RPC-style messages. Its key architectural lesson is the protocol shape; EOS should not copy that protocol without a specific integration requirement.

Its item lifecycle is explicit: item/started, optional typed delta events, and item/completed, inside turn/started and turn/completed. The server can issue a command-approval request and pause the turn until the client answers. Clients must tolerate additive events, partial streams, reconnect, and replay; typed bindings can be generated from the protocol schema.

Design continuity from the start

• Reuse the workspace or container when dependency state and intermediate artifacts need to persist.
• Store handoff artifacts in a known, durable location.
• Compact conversation state proactively while preserving objectives, decisions, results, and pending work.
• Make operations idempotent or resumable where possible.
• Stream progress as structured events rather than parsing terminal prose.
• Define cancellation, timeout, retry, approval, and reconnect behaviour.
• Pin or negotiate runtime versions so client and server capabilities remain compatible.

Make the system legible to the agent

Expose the same signals a human engineer would use: rendered UI, logs, metrics, test failures, dependency graphs, architecture boundaries, and current plans. Prefer concise, queryable interfaces to screenshots or unbounded log dumps.

As throughput rises, encode architectural constraints and cleanup processes in the harness. Fast generation without automated review, dependency rules, documentation checks, and periodic garbage collection merely produces drift faster.

A strong change environment supports this evidence loop:

1. validate a clean baseline;
2. reproduce and record the failure;
3. implement the change;
4. rerun the application and quantitative checks;
5. record the resolution;
6. handle review and build failures; and
7. escalate only the decisions that require human judgement.

OpenAI’s case study used one application and telemetry stack per worktree, with browser control plus queryable logs, metrics, and traces. The transferable standard is isolated, repeatable, queryable feedback—not those particular products.

Pick the lightest integration surface

Use a one-shot CLI for CI or bounded automation, a language SDK for embedded server workflows, a full event protocol for interactive and resumable clients, and MCP when the agent should appear as a callable tool in a broader system. Cross-provider abstractions improve portability but may hide richer lifecycle, diff, or approval semantics; choose that trade-off deliberately.

Do not copy a high-throughput team’s relaxed merge policy with its harness. Blocking gates should reflect failure impact, rollback cost, signal quality, regulation, and demonstrated recovery capability.