How to Inventory AI Systems in a Hospital: Step-by-Step

Why an Accurate Inventory Is a Prerequisite for Governance

Hospitals deploy AI across clinical decision support, revenue cycle automation, scheduling, and increasingly through AI agents that interact directly with EHR and operational systems. Many of these tools enter the organization through departmental purchases, vendor updates to existing platforms, or pilot programs that never transition to formal IT oversight. The result is a gap between the AI systems actually running in the environment and the systems documented in existing IT asset records.

This gap has direct compliance consequences. The HIPAA Security Rule requires covered entities to conduct an accurate and thorough risk analysis that identifies systems creating, receiving, maintaining, or transmitting electronic protected health information. An AI tool operating without documentation cannot be included in that risk analysis, which leaves the analysis incomplete regardless of how thorough the surrounding process appears. The NIST AI Risk Management Framework treats system inventory as a foundational input to its Map function, the step that establishes context before any risk measurement or control implementation occurs. Without an inventory, later governance activities such as access control, incident response, and regulatory reporting have no reliable foundation to draw from.

Categories of AI Systems Operating in a Hospital

Before building an inventory, governance teams need a working definition of what counts as an AI system in this context. Three categories typically apply.

Clinical decision support tools include imaging triage systems, sepsis prediction models, and other algorithms that influence diagnostic or treatment decisions. Many of these are embedded within certified EHR platforms rather than deployed as standalone applications, which means they will not surface through conventional software asset scans. ONC's HTI-1 rule requires certified health IT developers to disclose whether predictive decision support interventions are AI-based, giving hospitals a documented reference point for tools operating inside certified EHR modules.

Administrative and operational automation covers scheduling systems, billing and coding assistants, and robotic process automation bots that move data between systems. These tools often carry lower clinical risk profiles but can still process protected health information and warrant inclusion in the inventory for that reason alone.

AI agents represent a growing third category: software with system-level access that interacts with EHR data, lab systems, or scheduling platforms through APIs rather than a conventional user interface. Because these agents frequently exist as configurations, scripts, or integration logic rather than installed applications, they are the category most likely to be missed by standard software inventories.

Classification Criteria That Support Downstream Governance

Discovery produces a list. Classification turns that list into something governance, security, and compliance teams can act on. Four criteria are consistently relevant in hospital environments:

• Data sensitivity: what categories of PHI or operational data the system accesses.
• Patient impact: whether the system directly influences diagnostic or treatment decisions versus administrative processes.
• Autonomy level: whether the system operates under human review or takes action independently, a distinction the NIST AI RMF Playbook specifically recommends documenting.
• Integration points: which core systems, such as the EHR, PACS, or lab platforms, the AI tool connects to.

These tags should map to existing hospital risk classification schemes rather than introduce a separate taxonomy, since duplicate classification frameworks create reconciliation problems during audits and incident response.

Integrating the Inventory with Existing Governance Processes

• Keep the inventory inside existing ITAM/CMDB workflows rather than a standalone spreadsheet or parallel database.
• Tie ownership fields directly to incident response and access control processes, not just documentation.
• Treat the inventory as a recurring process rather than a one-time project, since embedded AI features change with vendor updates.
• Use classification tags to feed risk analysis and compliance reporting rather than storing them as static metadata.

AI Agents and the Limits of Traditional Discovery

AI agents complicate hospital inventories in ways clinical decision support tools generally do not. A diagnostic algorithm has a defined scope: it processes specific inputs and produces a specific output within a known clinical workflow. An AI agent with API-level access to the EHR, scheduling system, or lab platform can take actions across multiple systems, and its behavior may change as it is reconfigured or granted additional permissions over time. Traditional software asset management tools were not built to track this kind of behavior, since agents often exist as configuration objects or integration logic rather than installed applications.

This is where inventory work intersects with runtime governance. An accurate inventory establishes what exists at a given point in time; ongoing oversight of agent identity, permissions, and tool access determines whether that inventory stays accurate as agents are modified or granted new capabilities. Trussed AI provides runtime governance and security controls for AI agents, including agent identity management, permission and least-privilege enforcement, tool approval workflows, and audit logging, which help ensure that the inventory reflects actual agent behavior rather than a snapshot that becomes outdated as soon as an agent's configuration changes.