Implementation Guide

    Stopping a runaway AI agent loop without losing completed work

    Contain a runaway agent in stages instead of killing the process. Apply rate limits and call or step budgets first, block or escalate specific tool calls next, and halt the run at a known-good point last. Completed work survives only if the application checkpoints state and makes tool calls idempotent. Runtime policy enforcement limits what the agent can do during the incident and produces the record of what it was permitted to do and did.

    Apply containment in layers

    1. 1

      1. Rate limit tool calls

      Cap calls per tool and per time window so a retry storm slows down and becomes visible before it exhausts anything.

    2. 2

      2. Enforce call and step budgets

      Set a maximum number of model calls, tool calls and planning steps per run. Exceeding a budget triggers a defined action, not silent continuation.

    3. 3

      3. Block the specific call

      Deny the repeating tool call or argument pattern while leaving the rest of the agent's permitted actions available.

    4. 4

      4. Escalate for human approval

      Route consequential or repeated actions to a named approver. The agent waits, and nothing further executes until a decision is made.

    5. 5

      5. Constrain or revoke authority

      Use short-lived, task-scoped credentials so access can lapse or be withdrawn mid-run.

    6. 6

      6. Stop at a checkpoint

      End the run only once state is persisted, so the work already done can be inspected and resumed deliberately.

    Evidence to retain for the incident review

    • The policy version in force and the thresholds that triggered containment
    • Each tool call and interaction, with the decision taken (allow, redact, block, rate limit, escalate), the time and the identity involved
    • Budget and rate-limit events, including which limit was reached and what followed
    • Approval requests, the approver, the decision and the timestamps
    • Completed tool calls with their idempotency keys, from the application's own checkpoint records
    • The point at which the run was halted and the state persisted

    What a runaway loop looks like, and why halting is not discarding

    A runaway loop is an agent run that keeps issuing model calls and tool calls without converging on a result. The usual patterns are retrying a failed action with the same arguments, re-invoking a tool whose output the agent does not recognize as success, re-planning repeatedly without progress, and running with no unambiguous termination condition. In multi-agent designs, variance compounds across agent boundaries, tool invocations and orchestration steps, so non-converging behavior is harder to predict than in a single-agent run.

    The root causes are mostly design decisions: no step or call ceiling, vague success criteria, tool errors that do not separate retryable from permanent failures, and tool access broader than the task needs. Treat the loop as a symptom of one or more of these, not as a random fault.

    Halting an agent means no new actions are taken. Discarding its work means earlier actions, their effects and their record are lost or left in doubt. A hard kill tends to do the second while appearing to do the first. It can leave partial writes, orphaned state and no reliable list of completed steps, and a restart can then duplicate side effects. The goal during an incident is to stop new actions while keeping everything already done intact and accounted for.

    What the application and agent framework must own

    Runtime policy can stop an action. It cannot make a half-finished workflow coherent. That depends on how the application is built.

    Checkpointing: persist agent state at defined boundaries, such as after each completed tool call, so a halted run has a last known-good point. Resumable state: store enough to continue from that point without replaying earlier steps, including which steps finished and which were in flight. Idempotent tool calls: give each side-effecting call a stable key, so a retry or a resume does not repeat a write, a message or a payment. Where a downstream system cannot honor idempotency keys, wrap it so the application checks for a prior result first.

    Termination conditions: define success, failure and give-up explicitly. Classify tool errors as retryable or permanent, and cap retries with backoff. A run that cannot tell it has finished will keep going, and no external control fixes that design gap.

    The tradeoff is overhead. Checkpointing after every call adds latency and storage, while checkpointing too rarely widens the window of unrecorded work. Choose boundaries by the cost of redoing or reversing the step. Check how your chosen framework implements these capabilities before relying on them, because behavior differs between tools.

    What runtime policy enforcement contributes, and where it stops

    Enforcement in the execution path acts on individual interactions and tool calls. At that scope it can allow a call, redact data in it, block it, rate limit it, or escalate it for human approval. Because the decision happens as the call is made, a loop can be constrained while it is running, not reviewed after the fact. This matters because authorization at task start is not enough. NIST's agent identity guidance notes that without runtime verification, an agent holding a valid token cannot be interrupted if it is hijacked or manipulated mid-task. It also recommends zero standing privilege, least-privilege scope and ephemeral access, so authority can be contained or revoked when anomalous behavior is detected.

    Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants, at the level of individual interactions and tool calls. Rate limits, blocks and escalation for human approval are applied there, and the enforcement itself generates audit-ready evidence.

    The limits are as important. A per-call control sees calls, not the intent of the whole workflow. It does not decide whether a task is logically complete, and it does not restore agent state. Trussed does not provide end-to-end tracing of agentic workflows or workflow-level governance, so checkpointing, idempotency and resumption remain the framework's job. The two layers are complementary: the application makes a stop safe, and runtime enforcement makes a stop possible and provable.

    Criteria for evaluating runtime enforcement

    If you assess a runtime enforcement layer for this use case, ask for a demonstration of each criterion below rather than accepting a description. The audit-ready evidence Trussed generates is mapped to HIPAA, NAIC, NYDFS, EU AI Act and NIST requirements. Confirm that any layer you evaluate states its evidence content and retention in comparable terms.

    First, does it act per interaction and per tool call, with allow, redact, block, rate limit and escalate available as distinct outcomes? Second, can escalation be driven by thresholds you define and routed to the right approver? Third, can authority be constrained or revoked mid-run without depending on a long-lived token? Fourth, what does it record about policy applied, actions allowed or blocked and approvals requested, and where is that evidence stored? Trussed supports self-managed deployment in the customer cloud, with configurable retention and data residency. Fifth, which recovery responsibilities does it leave with the agent framework? A clear answer to the last question is a sign the boundary has been thought through.

    Graduated containment

    Constrain

    Rate limits and call or step budgets slow and cap the loop.

    Block or escalate

    Specific tool calls are denied or sent for human approval.

    Halt at a checkpoint

    The run stops at a known-good state, not mid-write.

    Review

    Evidence shows what was allowed, blocked, approved and completed.

    See how enforcement and evidence work at the tool-call level

    Review how policy is enforced in the execution path and what audit-ready evidence a loop incident leaves behind.

    See How Evidence Is Produced
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.