LLM Governance Requirements by Industry: Healthcare, Insurance, and Banking

Healthcare as the reference regime

HIPAA is the most useful starting point because its obligations attach to the data rather than to the technology. The Security Rule requires covered entities and business associates to ensure the confidentiality, integrity, and availability of all ePHI they create, receive, maintain, or transmit. The word "transmit" matters for LLM use. A practical reading is that prompts, model outputs, and tool-call payloads containing ePHI fall within that scope, whether the model is internal or hosted by a vendor.

The Technical safeguards section of the Security Rule, 45 CFR 164.312, includes the audit controls provision at 164.312(b). It is the natural anchor for deciding what evidence an LLM deployment should produce. Enforcement evidence supports an audit controls position. It does not by itself establish compliance, which is a judgment the regulated entity makes across its whole program.

The Security Rule evaluation standard adds a lifecycle duty. When environmental or operational changes affect the security of ePHI, regulated entities must perform technical and non-technical evaluations of their safeguards and document compliance. Introducing an LLM that touches ePHI is such a change. That evaluation is a documentation task, not a runtime control.

The Privacy Rule contributes two further points. Covered entities must have written contracts or other arrangements with business associates. A cloud service provider that receives and maintains only de-identified information is not a business associate under the HHS FAQ, so each LLM vendor's data handling determines its status.

The three regimes side by side

The instruments below differ in scope and in how they are written. They are not interchangeable, and this comparison covers the provisions that bear most directly on LLM use.

• Healthcare: HIPAA Security and Privacy Rules: Focus: ePHI confidentiality, integrity, and availability; audit controls; business associate arrangements. Organizational duties: documented evaluations after operational change, sanctions for workforce non-compliance. Runtime relevance: high, because obligations attach to data in transit and use.
• Healthcare: ONC HTI-1 (89 FR 1352): Focus: governance policies and implemented controls for Health IT Modules under 45 CFR 170.315(b)(11)(vi)(C). Applies to health IT developers, not to every healthcare LLM user.
• Insurance: NY DFS Insurance Circular Letter 7 (2024): Focus: a corporate governance framework overseeing use of ECDIS and AIS in underwriting, scaled to the insurer's nature, scale, and complexity, with attention to third-party vendor systems. Mostly organizational, with a risk-based standard.
• Banking and financial services: 23 NYCRR 500 as applied to AI: Focus: risk assessment and minimum cybersecurity standards for AI-related threats; third-party service provider policies covering access controls, encryption, due diligence, and contracts; data governance across collection, storage, processing, and disposal. Mixed runtime and organizational.

What the regimes share and where they diverge

Three themes recur. The first is protection of sensitive data: ePHI under HIPAA and nonpublic information (NPI) under the NYDFS guidance, which also asks for data governance procedures covering the full data lifecycle. The second is access control, which NYDFS names as a minimum for third parties that access information systems or NPI. The third is third-party oversight. It appears in NYDFS cybersecurity guidance, in the New York insurance letter's attention to vendor systems, and in HIPAA's business associate arrangements. The same theme is expressed through different instruments with different mechanisms.

The differences are in the object of regulation. HIPAA regulates a data class, so any LLM that touches ePHI is in scope. The insurance letter regulates a use case, underwriting and pricing, and expects a governance framework proportionate to the insurer. NYDFS regulates cybersecurity risk at covered entities and treats AI as a source of threat. Third-party posture also differs. The HHS FAQ states that covered entities are not required to monitor how business associates carry out privacy safeguards under the Privacy Rule. NYDFS, by contrast, calls third-party service provider policies and due diligence among the most important requirements for AI-related risk. An organization may choose stricter internal oversight than HIPAA demands, but that is a policy choice.

Practical next steps and where Trussed fits

Start by determining, for each LLM deployment, whether ePHI or NPI can reach the model or its tools, and whether each vendor is a business associate. Record the safeguard evaluation that follows from that change. Then separate the resulting requirements into the two columns above, so that each has a named owner and a named form of evidence. Insurers and financial services firms can follow the same sequence, replacing the HIPAA anchors with their own instruments.

Trussed enforces policy in the execution path across models, agents, tools, and MCP servers. It applies allow, redact, block, rate limit, and escalate for human approval to individual interactions and tool calls, and it produces audit-ready evidence mapped to HIPAA, NAIC, NYDFS, and NIST. It runs in production with healthcare and pharma organizations and can be self-managed in the customer cloud. It addresses the runtime column only. Organizational governance, vendor due diligence, and documented evaluations remain with the regulated enterprise.