HIPAA Periodic Evaluation for Deployed AI Systems Using Runtime Evidence

What the evaluation and audit controls provisions require

Section 164.308(a)(8) requires a covered entity or business associate to perform a periodic technical and nontechnical evaluation. The evaluation is based initially on the standards implemented under the rule, and afterward in response to environmental or operational changes affecting the security of ePHI. Its purpose is to establish the extent to which the entity's security policies and procedures meet the requirements of the subpart. The preamble to the 2003 final rule adds that entities must assess the need for a new evaluation when their security environment changes, for example when new technology is adopted or new risks are recognized. It also says the evaluation may be performed by the entity's own workforce or by an external party.

Section 164.312(b) requires hardware, software, or procedural mechanisms that record and examine activity in information systems that contain or use ePHI. Section 164.312(a)(1) limits access to ePHI systems to persons or software programs that have been granted access rights. An AI agent or tool integration is reasonably read as a software program under that language, though that is interpretation rather than explicit text.

Neither provision names AI, prompts, agents, or tool calls. Applying them to deployed AI is an interpretation of technology-neutral text. HHS stated in the preamble to its January 2025 proposed Security Rule amendments that ePHI in AI training data, prediction models, and algorithm data is protected by the HIPAA Rules. As of November 2026, this guide treats that proposed rule as a proposal and not as a requirement.

Why point-in-time evidence falls short for deployed AI

A questionnaire, a pre-deployment test result, or an annual assessment describes a system at one moment. A deployed AI system does not stay in that state. Models are swapped, prompts are edited, agents gain tools, and connected data sources change. Each of these can be an operational change that, under 164.308(a)(8), calls for a new evaluation.

The gap is between what the system was designed to do and what it was allowed to do and did with PHI in production between evaluations. Compliance teams often hold the first and lack the second. When an evaluator or auditor asks what a given AI interaction was permitted to do, the answer should not depend on engineers piecing together application logs, prompt histories, and tool records from several systems after the fact.

This is the core distinction in this guide. Evidence generated at the point where policy is enforced in the AI execution path records the decision as it happens. Evidence reconstructed later records someone's best reading of what probably happened. The first supports a periodic evaluation directly. The second makes the evaluation a forensic exercise.

Runtime evidence categories that support an evaluation

The mapping below is practitioner interpretation. The Security Rule does not prescribe evidence formats, and OCR does not prescribe a particular evaluation or risk management methodology. These categories are a reasonable way to answer the two questions an evaluation needs to settle: what the AI was allowed to do, and what it did with PHI.

Runtime evidence has limits. It shows activity that passed through the enforcement point. It does not by itself establish the nontechnical safeguards that 164.308(a)(8) also covers, such as policies, workforce training, and procedures.

• Policy decisions: Each allow, redact, block, or escalate decision, tied to the policy version in force. This shows what the system was permitted to do.
• PHI handling actions: Records of what PHI was redacted, blocked, or passed through in an interaction. This supports the audit controls review.
• Human approval escalations: Who was asked to approve, what was decided, and when. This shows where human oversight operated in practice.
• Tool call outcomes: Whether a tool call was permitted, what it accessed, and what resulted. This addresses access by software programs.
• Retention and residency settings: The configured retention period and data location for evidence and interaction data, as they stood during the review period.
• Change context: The model, prompt, tool, data source, and policy version active at the time, so changes since the last evaluation can be identified.

Where enforcement in the execution path fits

Trussed enforces policy in the execution path across models, agents, tools, and MCP servers, scoped to individual interactions and tool calls. It can allow, redact, block, rate limit, or escalate for human approval. Because policy is enforced at runtime, not reviewed after the fact, the enforcement itself generates the evidence: continuous, audit-ready evidence mapped to HIPAA, including PHI handling and 164.312(b) audit controls. Deployment is self-managed in the customer cloud, with configurable retention and data residency. Trussed runs in production with healthcare and pharma organizations.

This addresses the technical evidence an evaluation draws on. It does not replace the nontechnical review, the evaluation itself, or the judgment of your compliance team.