Implementation Guide | Banking

Running a Mock AI Examination at a Bank

A mock AI examination is an internal rehearsal in which the bank plays the examiner. You define scope and applicable supervisory expectations, issue an examiner-style request list, test whether evidence can be produced and is accurate, interview owners, rate the gaps, and track remediation. The exercise tests evidence production, not policy quality. For AI agents, that includes identity, permissions, tool-call logs, and policy enforcement records.

Run the exercise in six steps

The mock examination follows one sequence, from scoping through remediation. The table summarizes each stage; the sections that follow explain how to carry them out.

Mock examination flow
StepStageWhat happens
1ScopeAI systems, generative AI, and agents mapped to the frameworks you will test against.
2RequestExaminer-style request list issued with deadlines.
3TestEvidence checked for completeness, accuracy, and timeliness.
4InterviewOwners walk through live use cases.
5RateFindings rated on a severity scale defined in advance.
6RemediateOwners, dates, and retesting tracked to closure.

Define scope and the frameworks you will test against

Start by defining scope and the applicable supervisory expectations. Scope covers AI systems, generative AI, and AI agents, each mapped to the frameworks you will test against.

Agree on this mapping before any requests go out. It sets the boundaries of the exercise and gives every later finding a clear reference point.

Assemble the team and assign roles

Because the bank plays the examiner, the exercise needs people in examiner-style roles who issue requests, test evidence, and rate findings. Owners of the AI systems and use cases are the counterparty: they produce evidence and are interviewed.

  • Examiner team: issues the request list, tests the evidence, and rates the gaps.
  • Owners: produce the requested evidence and walk through live use cases.
  • Remediation owners: take named responsibility, with dates, for each finding.

Build the request list and test the evidence

Issue an examiner-style request list with deadlines. The deadlines matter because the exercise measures whether evidence can be produced when asked.

Check each item three ways

  • Completeness: the evidence covers what was requested.
  • Accuracy: the evidence reflects how the system actually operates.
  • Timeliness: the evidence arrives within the deadline set.

Interview the owners

After testing the evidence, have owners walk through live use cases. The interviews show whether what the owners describe matches what the evidence shows.

Probe runtime and audit evidence for AI agents

For AI agents, the request list should reach beyond documentation into runtime and audit records. The exercise should confirm that the bank can produce each of the following:

  • Identity: records showing which agent acted.
  • Permissions: records showing what each agent was allowed to do.
  • Tool-call logs: logs of the tools each agent invoked.
  • Policy enforcement records: records showing that policies were enforced at runtime.

If any of these cannot be produced, accurately and on time, treat it as a gap and carry it into the rating step.

Rate findings and track remediation

Rate findings on a severity scale defined in advance, so that ratings are consistent and are not negotiated after the fact.

Then track remediation to closure. For each finding, record the owner, the target date, and the retest that confirms the gap is closed.

Plan your evidence for AI agents

If your mock examination exposes gaps in agent identity, permissions, or audit logging, talk with Trussed AI about runtime governance for enterprise AI agents.

Talk to an Expert