Implementation Guide

    How to prove Shadow AI is under control to bank internal audit

    Internal audit needs two kinds of proof for Shadow AI. Design evidence shows the scope, policy, ownership and risk mapping. Operating-effectiveness evidence is a sampleable population of events showing what AI was allowed to do, what was redacted, blocked or escalated, and who approved exceptions. Policies, attestations and discovery scans show intent. Policy enforced in the execution path produces the event-level records an auditor can test.

    Evidence required at each stage of the audit lifecycle

    1. 1

      1. Scoping

      A defined Shadow AI population and sampling frame covering public AI sites, coding assistants, unapproved models, and unvetted tools and MCP servers, including external AI touchpoints.

    2. 2

      2. Control definition

      Control narratives with objectives, owner, risk mapping and the actions available: allow, redact, block, escalate for human approval.

    3. 3

      3. Walkthrough

      A live trace of one interaction from request to policy decision to retained record, including an escalation and its approver.

    4. 4

      4. Sampling and testing

      A complete event population with timestamps, policy version, decision and outcome, from which auditors select samples and re-perform the check.

    5. 5

      5. Findings and remediation

      Documented exceptions, owners, policy or permission changes linked to testing and approval evidence, and a retest record.

    • Completeness: every in-scope interaction and tool call produces a record, and the coverage boundary is documented.
    • Immutability: records cannot be altered by the teams whose activity they describe.
    • Retention: periods match audit and examination cycles, and are configurable.
    • Data residency: records stay where policy and regulators require, with sensitive content protected.
    • Traceability: each record names the policy version, decision, and any human approver.
    • Control mapping: records roll up to named control objectives and to the framework the bank uses.
    • Can you show, for a sampled event, what the AI was allowed to do and which policy applied?
    • Is there an independent record of each exception approval, not a reconstruction from separate systems?
    • Does the control population cover coding assistants, unapproved models, and MCP servers or tools, with completeness demonstrated?
    • Are policy, tool and permission changes documented and linked to testing and approval evidence?
    • Are retention, immutability and residency specified and mapped to named control objectives?
    • Is the third-party AI tool population covered by contractual audit rights and review of vendor assurance results?

    What internal audit actually tests

    Internal audit does not accept the existence of a policy as proof. SR 11-7 states that internal audit should verify that acceptable policies are in place and that model owners and control groups comply with them. It should also verify records of model use and validation, and test whether procedures are documented and carried out as specified. The revised guidance in SR 26-2 says internal audit would generally not duplicate model risk management activities and instead evaluates whether those practices are rigorous and effective and whether related policies are implemented accordingly. As of November 2026, SR 26-2 is expected to be most relevant to Federal Reserve-regulated banking organizations with over $30 billion in total assets.

    The OCC's 2026 supervisory framework for bank AI use adds an assurance framing. It defines assurance as evidence that AI system or agent controls continue to operate effectively, and it groups testing, documentation and auditability, third-party risk, and internal audit and independent challenge under that heading.

    For Shadow AI, three concepts follow. Design effectiveness asks whether the control, as written, would address the risk: is the scope complete, are the rules specific, is ownership clear. Operating effectiveness asks whether the control acted as designed across a period, which requires a population of events from which an auditor can draw a sample. Evidence sufficiency asks whether those records are complete, tamper-resistant, retained long enough and traceable to a control objective. Most Shadow AI programs can show the first. Few can show the second and third.

    What each control type gives an auditor

    Each common Shadow AI control produces a different kind of evidence, and each has a limit. The mapping below is an analytical view of how an auditor is likely to weigh them, not a regulatory classification.

    • Discovery scans: Evidence: a point-in-time inventory of tools and users. Limit: shows what existed on the scan date, not what the AI was allowed to do or did afterward.
    • Policy and attestations: Evidence: design intent and management assertion. Limit: no operating-effectiveness data, and attestations rely on self-reporting.
    • Network-level blocking: Evidence: a block list and denied-connection counts. Limit: coarse, with no view into the interaction, the data sent or the tool call requested.
    • After-the-fact log review: Evidence: records reconstructed from several systems. Limit: completeness is hard to demonstrate, and review happens after the exposure.
    • Runtime policy enforcement: Evidence: a per-event decision (allow, redact, block, escalate) tied to the policy that applied. Limit: only covers traffic that passes through the enforcement point, so scope must be shown.

    Enforced at runtime, not reviewed after the fact

    The core distinction is where the control sits. An attestation or periodic review asks people to report on behavior. A control in the execution path decides each request before it completes, and the decision itself becomes the record. That is why enforcement in the path generates continuous evidence as a by-product, while review after the fact has to reconstruct it. The sources do not prescribe this architecture, but it fits the evidence expectations above: event-level records, linked to approvals, produced continuously.

    The actions matter for design testing. Allow, redact and block are deterministic and can be re-performed against a policy. Escalation for human approval gives meaningful review a defined trigger, and the approver's decision becomes part of the record. Rate limits and tool-level permissions help bound what an agent or coding assistant can do, which matters when the question is what AI was allowed to do, not only what it did.

    Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants, at the level of individual interactions and tool calls. Enforcement produces audit-ready evidence mapped to frameworks including NIST, and deployment is self-managed in the customer cloud with configurable retention and data residency. This has run in continuous production in a regulated enterprise for more than 18 months. Among regulated financial sectors, insurance is where Trussed is deepest.

    Whoever operates the control should not be the only party who can validate it. The OCC framework notes that effective challenge may depend on independence from both the owning and validating groups. Assign Shadow AI control ownership to the first line, validation to the second, and testing to internal audit, and keep their evidence access separate.

    From Shadow AI discovery to audit-ready evidence

    Scope

    Define the AI population: public AI sites, coding assistants, unapproved models, unvetted tools and MCP servers.

    Policy

    State control objectives and map each to a risk and an owner.

    Enforce

    Apply allow, redact, block or escalate in the execution path.

    Evidence

    Retain event-level records linked to the policy and approval that applied.

    Test

    Let internal audit sample events and verify them independently.

    Build an evidence package auditors can sample

    See how enforcement in the execution path produces event-level, audit-ready evidence for Shadow AI controls.

    See How Evidence Is Produced