All articles Agents

Agent Harnesses: The Engineering System Around Reliable AI Agents

A technical guide to building agent harnesses: the runtime, state, tools, policies, evaluation, and observability that turn an LLM loop into a dependable system.

· 6 min read · By AI AgentsAgent HarnessLLM SystemsTool Use

An LLM call is not an agent. An agent is a policy-driven system that observes state, chooses an action, invokes a tool or produces an answer, and then updates its state. The agent harness is the engineering layer that makes that loop bounded, inspectable, recoverable, and testable.

What an agent harness controls

The model supplies a probabilistic decision. The harness supplies the contract around that decision. At minimum, it owns:

  • Execution: retries, timeouts, cancellation, concurrency, and step limits.
  • State: the run identifier, messages, tool results, intermediate artifacts, and checkpoint history.
  • Capabilities: the tools the model may call, their schemas, permissions, and side effects.
  • Policy: what is allowed, what needs human approval, and what must be blocked.
  • Evidence: traces, inputs, outputs, citations, tool calls, and cost data.

This separation matters. Prompt instructions are not an access-control system, and a tool description is not a transaction boundary. The harness must enforce invariants outside the model.

The execution loop

A useful abstraction is a state machine rather than an unconstrained while-loop:

RUNNING -> MODEL_DECISION -> TOOL_PENDING -> TOOL_RESULT -> RUNNING
                 |                                  |
                 +------------ FINAL ---------------+
                 +------------ BLOCKED / APPROVAL --+
                 +------------ FAILED --------------+

At each step, the harness sends a bounded view of state to the model. The model can either return a final message or a structured tool call. The harness validates the call, checks policy, executes the tool, records the result, and continues.

A simplified control flow looks like this:

async def run_agent(run, request):
    state = await store.load(run.id)
    for step in range(run.max_steps):
        context = context_builder.build(state, token_budget=run.context_budget)
        decision = await model.decide(context, tools=policy.allowed_tools(state))
        audit.record_decision(run.id, step, decision)

        if decision.final_answer is not None:
            return finalize(decision.final_answer, state)

        call = tool_registry.validate(decision.tool_call)
        policy.check(call, state)                 # may return APPROVAL_REQUIRED
        result = await executor.invoke(call, timeout=call.timeout)
        state = state.apply(call, result)
        await store.checkpoint(run.id, state)

    raise StepBudgetExceeded(run.id)

The important detail is not the syntax. It is that every transition is explicit and durable. A process crash after a payment tool succeeds must not cause the harness to repeat the payment blindly.

State, memory, and context

Agent state should be divided by lifetime and trust level:

  • Run state: the current task, plan, tool outputs, and status. It belongs to one execution.
  • Conversation memory: user-approved facts that may be reused in later runs.
  • Knowledge: retrieved documents or database records. These are evidence, not instructions.
  • Control state: budgets, approvals, idempotency keys, policy decisions, and checkpoint versions.

Do not concatenate all of these into one undifferentiated prompt. Retrieved text may contain prompt injection. A safer context builder gives each source a separate envelope and tells the model what it is allowed to influence. The harness still validates the result independently.

Context is a constrained resource. A practical policy is to reserve tokens for the answer and tool arguments, then allocate the remainder across recent turns, task state, retrieved evidence, and summaries. Summarization should preserve decisions, unresolved questions, identifiers, and citations; deleting those fields makes the agent appear coherent while silently losing state.

Tools as typed capabilities

A tool should look more like a small API than a paragraph of instructions. Its schema should define required fields, ranges, enums, and whether the operation is read-only or mutating. The executor should apply server-side authorization using the run identity, not a user-provided argument.

approve_invoice(
    invoice_id: UUID,
    approval_reason: str,
    idempotency_key: str
) -> ApprovalReceipt

Tool design follows a useful risk gradient:

  • Pure reads can often execute automatically.
  • Reversible writes should be logged and rate-limited.
  • Irreversible or financial actions need an approval state and an idempotency key.

Idempotency is essential. A retry after a network timeout must resolve to the original receipt, not create a second side effect. Store the key with the operation result and make the database constraint enforce uniqueness.

Budgets, policy, and approvals

Reliable agents fail closed at the boundaries. Use several independent budgets:

  • Step budget: prevents loops and runaway planning.
  • Wall-clock budget: bounds user-visible latency.
  • Token budget: controls context growth and spend.
  • Tool budget: limits expensive or sensitive capabilities.
  • Money and rate budgets: limits cumulative side effects per run.

Policies should be deterministic wherever possible. For example, a payment above a threshold should always enter APPROVAL_REQUIRED, regardless of how persuasive the model's reasoning sounds. The approval record should include the exact arguments, evidence references, policy version, and the identity of the approver.

Prompt injection defense is layered: isolate untrusted content, strip or mark instructions from retrieved documents, restrict tool permissions, validate output schemas, and require confirmation for high-impact actions. No single classifier can replace these controls.

Evaluation and observability

Agent evaluation must measure more than final-answer similarity. Track:

  • Task success: did the requested outcome occur?
  • Tool correctness: were the right tools called with valid arguments?
  • Trajectory efficiency: steps, retries, latency, and tokens.
  • Grounding: can claims be traced to retrieved evidence or tool results?
  • Safety: did the run violate a policy or attempt an unauthorized side effect?

Every run should produce a trace with a stable run ID, model version, prompt-template version, tool calls, input and output hashes, latency, token counts, policy outcomes, and checkpoint IDs. Redact secrets before traces leave the trust boundary. Sampling alone is not enough for high-impact tools; log every decision that can change external state.

Build replayable tests from recorded states. A deterministic fake tool layer lets you test planning and policy without calling production systems. For model changes, compare success, cost, latency, unsafe-action rate, and trajectory shape against a fixed evaluation set.

A practical architecture

A production harness commonly contains these components:

  • Run manager: creates runs, enforces budgets, and exposes cancellation.
  • Context builder: selects and labels state, memories, and evidence.
  • Model gateway: handles routing, structured output, timeouts, retries, and fallbacks.
  • Tool registry and executor: validates schemas, checks permissions, and invokes isolated capabilities.
  • Policy engine: evaluates deterministic rules and creates approval requests.
  • Checkpoint store: persists state transitions and idempotency records.
  • Trace and evaluation pipeline: records evidence and turns runs into regression tests.

The design goal is not to make the model deterministic. It is to make the system's consequences deterministic enough to govern. Once state, tools, policies, and evidence have explicit contracts, model improvements become measurable engineering changes instead of prompt folklore.

An agent harness is the difference between an LLM that can act and an AI system that can be trusted to act.

← Back to all articles