Implementation Guide

    Supervising AI Assistants for Client Account Statement Summaries

    Supervise these assistants by enforcing policy in the execution path on every interaction, not by sampling outputs afterward. Define what data and tools the assistant may use, map each risk to a policy with a default action (allow, redact, block, or escalate for human approval), and record each decision as evidence a reviewer or auditor can read. Runtime enforcement then sits alongside, not in place of, your existing supervisory procedures.

    Sequence the implementation

    1. 1

      1. Inventory and scope

      List assistants, data sources, tools, and channels. Mark which interactions provide information to clients.

    2. 2

      2. Map risks to policies

      Assign each risk class and control point a written policy, an owner, and a default action.

    3. 3

      3. Define escalation criteria

      Set content, sensitivity, and numerical-match rules for human approval, and name the approvers.

    4. 4

      4. Specify the evidence record

      Agree the fields, retention, and residency with compliance, legal, and security.

    5. 5

      5. Enforce in a limited pilot

      Run the policies in the execution path on one assistant and channel, with conservative escalation.

    6. 6

      6. Review and tune

      Supervisors review enforcement records, adjust thresholds, and extend scope on a cadence the firm sets.

    What you are supervising

    An assistant that summarizes account statements and performance does more than generate text. It retrieves sensitive account data, may call tools to fetch it, and produces statements about balances, returns, and holdings that a client may rely on. Each of those steps is a point where supervision can succeed or fail, so the unit of supervision should be the individual interaction and the individual tool call.

    Scoping starts with a simple question: which interactions provide information to a client? The SEC's 2023 proposed rules on predictive data analytics conflicts are a useful reference for the framing. They define an investor interaction to include providing information to an investor. They remain a proposal, and as of November 2026 they should not be treated as current law. Obligation-level questions about supervision, recordkeeping, and communications should be answered by your own compliance and legal functions. The design work below holds regardless of how those answers land.

    NYDFS also issued guidance in October 2024 on cybersecurity risks arising from AI. Entities it applies to should read the primary document and map it to their assistant inventory.

    Map each risk to an enforceable policy

    Three documented LLM risk classes cover most of the exposure. Prompt injection can lead to unauthorized access and compromised decision-making. Sensitive information disclosure can affect both the model and its application context. Misinformation means the model produces incorrect statements that users may act on. Numerical misstatement, unsupported performance claims, and omitted disclosures are best treated as forms of the misinformation class. That mapping is an editorial judgment, so state it explicitly in your policy documentation.

    The useful design principle is a two-way trust boundary. Neither what enters the model nor what leaves it can be inherently trusted. Statement text and client input should be checked on the way in, and the summary should be checked on the way out. A policy that exists only in the system prompt is a request to the model, not an enforced control.

    Separate the policies by control point so each has an owner and a default action.

    • Data access: Limit retrieval to the authenticated client's accounts. Block requests that reach beyond that scope, including those induced by text inside a statement.
    • Tool use: Allow only named tools with permitted parameters. Rate limit repeated retrievals.
    • Summary content: Redact identifiers that should not appear in the output. Block summaries that contradict retrieved figures.
    • Performance claims: Permit statements that restate reported returns. Escalate or block projections, comparisons, and characterizations the firm has not approved.
    • Client-facing delivery: Control which channels may receive an unreviewed summary and which require approval first.

    Decide when to allow, redact, block, or escalate

    Each interaction needs a deterministic outcome. Allow suits low-risk summaries where every figure traces to retrieved statement data and no restricted content appears. Redact suits outputs that are otherwise acceptable but contain data the client channel should not carry, such as full account numbers. Block suits clear violations: out-of-scope data access, disallowed tool calls, or instructions that attempt to override policy. Escalate suits cases where a person must decide.

    The criteria for escalation are a firm decision, because there are no standard thresholds. Reasonable inputs include content type (a returns narrative versus a balance recap), data sensitivity, whether a figure in the summary can be matched to the source data, and whether the output contains language that resembles a recommendation or forecast. Start conservatively. Route more to human approval at launch, then narrow the escalation rules as reviewers see which cases they approve unchanged.

    Two tradeoffs deserve attention. Heavy escalation protects clients but creates queues that delay delivery and tempt reviewers to approve by habit. Aggressive blocking reduces risk but can make the assistant unusable for routine requests. Track approval rates, overrides, and queue times as supervisory signals in their own right.

    Specify the evidence record before rollout

    Conventional application logs rarely map to a supervisory question. A reviewer asks what the assistant was permitted to do, what it did, and who decided. Design the record to answer those three questions for every interaction.

    At a minimum, capture the assistant and policy version in force, the data and tools requested and permitted, the decision taken at each control point (allow, redact, block, rate limit, or escalate), the output as delivered, and the reviewer identity, time, and outcome when a human approved or rejected it. Decide in advance how much raw content the record holds, since statement data is itself sensitive and retention and residency need to be configurable to your requirements.

    NIST AI RMF 1.0 is voluntary guidance, but it is a workable organizing reference for how records are grouped and reviewed. The NIST generative AI profile, still a draft in the material reviewed, points toward documentation that supports external scrutiny. Retention periods should come from your own recordkeeping obligations, not from the framework.

    Where runtime enforcement fits with existing procedures

    Runtime enforcement does not replace supervisory procedures. Principals still own review standards, exception handling, training, and escalation. What changes is timing and evidence. Policy is applied at the moment of each interaction, and the record is produced by that enforcement rather than reconstructed from samples later. Sampling remains useful as a quality check on the policies themselves.

    Trussed provides this layer. It enforces policy in the execution path across models, agents, and tools, with actions to allow, redact, block, rate limit, and escalate for human approval at the level of individual interactions and tool calls. Enforcement generates audit-ready evidence mapped to frameworks including NAIC, NYDFS, and NIST. It can be deployed self-managed in the customer cloud, with configurable retention and data residency. It does not provide transaction-level governance, model lineage, or end-to-end tracing of agentic workflows, so those needs, if you have them, require other measures.

    Whatever tooling you choose, test it against the same questions: can it act on both input and output, can it control tool calls, and does each decision record answer what was permitted, what happened, and who approved it?

    Supervision at each interaction

    Input and data access

    Check what statement data the assistant may retrieve before it reaches the model.

    Tool calls

    Permit only approved tools and parameters for each assistant role.

    Summary output

    Evaluate balances, returns, holdings, and performance language before delivery.

    Decision record

    Retain the policy applied, the action taken, and who approved it.

    Review your supervision model with an expert

    Walk through your assistants, policies, escalation criteria, and evidence requirements for client account summaries.

    Talk to an Expert
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.