How to Write a Health System AI Acceptable Use Policy

What an AI Acceptable Use Policy Must Do

A health system AI acceptable use policy is an operational governance artifact, not a static legal statement. Its purpose is to define, in terms that can be technically verified, which AI tools clinicians and staff may use, what data those tools may access, and what actions they may take on that data. This includes third-party generative AI applications, internal copilots embedded in clinical or administrative systems, and increasingly, agentic AI systems capable of taking autonomous actions such as querying records or calling external functions.

Without enforceable structure, health systems face inconsistent adoption across departments, unmanaged shadow AI use, and gaps against HIPAA and emerging state AI requirements. A policy that only states expectations, without a mechanism to verify compliance, does not reduce that risk in practice.

Regulatory Anchors That Shape Policy Boundaries

There is no single federal statute governing healthcare AI use. Health systems must synthesize existing HIPAA obligations with newer, largely voluntary AI-specific frameworks.

HIPAA's Security Rule technical safeguards (45 CFR 164.312) require access control, audit controls, and transmission security, all directly applicable to bounding AI usage on PHI. The minimum necessary standard (45 CFR 164.502(b)) constrains how much data an AI tool should ever be permitted to access. Any AI vendor that creates, receives, maintains, or transmits PHI on behalf of a covered entity is a business associate and requires an executed BAA under 45 CFR 164.502(e).

Beyond HIPAA, NIST's AI Risk Management Framework offers a Govern/Map/Measure/Manage structure useful for organizing policy sections, and its Generative AI Profile identifies risks such as data leakage and third-party component exposure relevant to vendor AI tools. HHS ONC's HTI-1 rule imposes transparency disclosure requirements on certified health IT that incorporates predictive decision support.

Federal direction on AI specifically has been unsettled since Executive Order 14110 was rescinded in January 2025, which increases the relative weight of state law. Colorado's AI Act, for example, imposes duties on developers and deployers of high-risk AI systems, including healthcare-related decision systems, effective February 2026. Policies should be written to accommodate jurisdiction-specific requirements rather than assuming a single national standard.

Structural Components the Policy Should Include

At minimum, an enforceable AI acceptable use policy should cover inventory, permitted use, technical enforcement boundaries, and review cadence. The following elements give reviewers a concrete checklist when drafting or updating the document.

• Approved tool inventory

  Documented AI tools with confirmed BAA status where PHI access exists.

• Permitted and prohibited use

  Explicit boundaries for clinical, administrative, and third-party AI use.

• Access and tool-call controls

  Role-based restrictions on data access and agentic AI actions.

• Audit and review cadence

  Logging sufficient to reconstruct AI-assisted decisions for compliance review.

Enforcing Policy Through Technical Controls

Written policy language has limited value if it cannot be technically enforced or audited. Role-based access controls should mirror the policy's defined user categories, restricting which AI tools and data sensitivity levels each role can reach. Approved enterprise AI tools, those with executed BAAs and logging in place, should be separated from unmanaged tools through network- or endpoint-level controls, so shadow AI use is limited by system design rather than by instruction alone.

Data flow controls should prevent PHI from reaching any AI model or vendor without a confirmed BAA in place. Audit logging should capture prompts, retrieved data sources, and outputs at a level of detail sufficient to reconstruct an AI-assisted decision for HIPAA accounting or internal compliance review, not just a record that a tool was accessed.

Addressing Agentic AI in Policy Language

AI systems capable of tool-calling or autonomous action introduce risks distinct from standard generative AI use. OWASP's guidance on large language model applications identifies excessive agency and insecure output handling as specific concerns for these systems, and NIST's Generative AI Profile recommends scoped permissions and oversight as a corresponding control.

In practice, this means tool-call permission scopes for agentic AI should be designed and approved before deployment, not addressed only through post-hoc policy language. A policy should specify that agentic systems are limited to a pre-approved allowlist of functions, that any expansion of those functions requires the same review as adding a new AI tool, and that autonomous actions are logged with the same rigor as data access events.

Monitoring for anomalous agent behavior, including prompt injection attempts, is a distinct technical control from standard access logging and remains an area without established healthcare-specific standards, which is a reason to build monitoring capacity into the policy's enforcement plan rather than treat it as optional.