What a Health System CIO Should Standardize First: Models, Policies, or Evidence

The decision and why it is hard

A health system CIO usually inherits AI that arrived through several doors: models selected by clinical or research teams, AI features embedded in vendor products, administrative tools, and coding assistants. Each team chose independently. The CIO has to decide where consistency goes first.

There are three candidate layers. The model layer is the set of models and AI-enabled products in use. The policy layer is the set of rules for what AI may do, especially with PHI, who may use it, and what needs human approval. The evidence layer is the record of what each AI system was permitted to do and what it did.

The footprint is already large and growing. ASTP/ONC data show 71% of hospitals reported using predictive AI integrated with the EHR in 2024, up from 66% in 2023. That figure covers EHR-integrated predictive AI only. Generative tools, coding assistants, and administrative features sit outside it, and they are often adopted without central selection.

Comparing the layers on four criteria

The criteria are rate of change, enforceability, auditability under HIPAA, and effect on clinical and operational teams. The assessments below reflect general technical reasoning about each layer.

• Models: Rate of change: highest, since models and vendor features are replaced or added on the vendor's schedule. Enforceability: low on its own, because choosing a model does not control what it does with PHI. Auditability: a model list does not show what happened in any interaction. Team effect: highest, because a mandated model removes tool choice from clinical and operational teams.
• Policies: Rate of change: lower, because rules about PHI handling, access, and approval do not depend on which model is used. Enforceability: high if enforced in the execution path, weak if kept as documents. Auditability: strong when enforcement is recorded, limited when policy is only written. Team effect: moderate, because teams keep their tools but work within common rules.
• Evidence: Rate of change: low, because the requirement is to record and examine activity, which does not change with the model. Enforceability: depends on a point where activity is captured. Auditability: directly what an auditor requests. Team effect: low, because teams do not change tools to produce it.

What HIPAA implies for the evidence layer

The audit controls standard at 45 CFR 164.312(b) requires hardware, software, and/or procedural mechanisms that record and examine activity in information systems that contain or use ePHI. The wording is technology-neutral. It does not name models or any particular evidence format, so an evidence layer defined around activity can remain constant while the systems producing that activity change.

Two neighboring standards matter for AI. The access control standard refers to allowing access only to persons or software programs that have been granted access rights, so software components fall within its wording. The integrity standard requires policies and procedures to protect ePHI from improper alteration or destruction. Counsel should decide how these apply to specific AI deployments, but a CIO can reasonably design for AI components that touch ePHI being in scope.

HHS proposed changes to the Security Rule in January 2025. In the preamble, HHS describes audit controls as crucial technical safeguards, says regulated entities must use their risk analysis and organizational factors to determine reasonable and appropriate controls, and says OCR enforcement experience suggests compliance could improve with more specificity. As of November 2026, treat this as a proposal and not as a current requirement. It does indicate the direction: evidence expectations are more likely to become more specific than less.

How the layers depend on each other

The layers are not independent, and the dependencies explain the order.

Evidence depends on policy. A record of activity is only useful to an auditor if it can be read against a statement of what was permitted. Without a defined policy, a record shows what happened but not whether it was allowed.

Policy depends on a point of enforcement. A written policy that is reviewed after the fact produces periodic reviews and samples. A policy enforced at runtime, in the execution path, produces a record at the moment each decision is made: allowed, redacted, blocked, or escalated. That record is a by-product of enforcement and does not need a separate reporting effort. This is why policy and evidence can be standardized without first choosing a model.

Models sit beneath both. If controls are built per model, every new model or vendor feature needs its own controls and its own evidence format, which produces the duplicated controls and uneven PHI handling that CIOs are trying to remove. If controls sit above the models, changing a model changes less.

This matches how ASTP/ONC SAFER Guide 4 treats change. It describes testing system and software updates and local customizations using a standardized approach, which standardizes the process around changing components and not the components themselves. The same guide places shared responsibility for AI safety on the adopting organization, including when AI is embedded in an EHR or vendor system. The policy layer therefore has to cover vendor-embedded features as well as models the health system selects.

Model standardization is still useful later. Evaluating models for performance, safety, and fairness remains a governance task. Once policy and evidence are common, the health system can compare models on the same terms and narrow its catalog on the basis of observed behavior.

What a first implementation phase looks like

A first phase can be limited to use cases that touch ePHI and should aim to produce one policy set and one evidence record that every AI use case can adopt.

• Inventory AI use cases and vendor-embedded features: Include clinical, administrative, and coding assistant use, and mark which touch ePHI. An inventory is a practice reference and not a HIPAA requirement.
• Map ePHI use cases to 164.312: Relate each to access control, audit controls, and integrity, and record the risk-analysis basis for the audit controls chosen.
• Write a common policy set: Cover PHI handling, permitted actions, and what requires human approval. Include policies for patient and clinician awareness of AI use before patient-care deployment, as SAFER Guide 4 describes.
• Define the evidence record: Specify what the record must show about what the AI was permitted to do and what it did. The fields are a design decision tied to your environment and risk analysis.
• Enforce policy at the point of use: Start with the highest-risk ePHI use cases. Distinguish policies enforced at runtime from those that depend on after-the-fact review.
• Define the model change process: Specify what is re-evaluated when a model changes or is added, and which controls and evidence formats are expected to stay the same.

Enforcing policy and producing evidence across many models

Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites, and coding assistants. Policy is enforced at runtime, not reviewed after the fact, and each interaction or tool call can be allowed, redacted, blocked, rate limited, or escalated for human approval. Enforcement generates continuous audit-ready evidence mapped to HIPAA and NIST. The NIST AI RMF is a voluntary framework, so this is framework alignment and not legal compliance.

The platform runs 100+ models under one policy plane and is in production with healthcare organizations. It can be self-managed in the customer's cloud, with configurable retention and data residency. Its scope is individual interactions and tool calls. That scope is what lets the policy and evidence layers stay constant while the model layer changes.