LangGraph vs LlamaIndex Workflows: Where to Enforce HIPAA Policy in Each Framework
Both frameworks give you places to write PHI checks: graph nodes and conditional edges in LangGraph, workflow steps and events in LlamaIndex Workflows. Those checks are application code, written per workflow, so they can drift and leave gaps in what you can show an auditor. A consistent HIPAA posture needs policy enforced at runtime in the execution path, with the framework constructs handling orchestration rather than serving as the only control.
Criteria for evaluating a runtime enforcement layer
- Acts independently of framework constructs, so one policy applies to LangGraph and LlamaIndex Workflows alike.
- Can allow, redact, block or escalate each model call, retrieval result and tool call, not only the final output.
- Acts on PHI before it enters model context, tool arguments or persisted state.
- Produces evidence per interaction in a consistent format, mapped to HIPAA technical safeguards including 164.312(b).
- Deploys in your cloud, with clear answers on whether ePHI leaves your environment and whether a BAA covers any third party.
- Lets you set retention and data residency for its own evidence, and defines human-approval rules once for all workflows.
The regulatory frame
The HIPAA Security Rule sets national standards for electronic protected health information (ePHI) that a covered entity or business associate creates, receives, uses or maintains. Section 164.312 sets the technical safeguards, and it applies in accordance with 164.306, the general rules for security standards. Audit controls sit in 164.312(b).
An agent that retrieves records, calls a model and invokes tools handles ePHI at several points, and each point is a place where a safeguard has to operate. In a proposed rule published in January 2025, HHS stated that ePHI in AI training data, prediction models and algorithm data maintained by a regulated entity for covered functions is protected by the HIPAA Rules. That is a statement in a proposal, not a final regulation, but it is a reasonable signal for scoping AI-related data stores.
Enforcement surfaces side by side
The table lists where a policy check can be written in each framework for each control point, and what the check cannot do on its own. The constructs named are the ones each framework exposes for composing workflows. In every row the check is code that a team writes and maintains inside the application.
- Model calls and PHI in prompts: LangGraph: a node or conditional edge before and after the model call. LlamaIndex Workflows: a step or event handler around the call. Limit: a check exists only where a developer placed it, and a new workflow starts without it.
- Retrieval and retrieved context: LangGraph: a node that runs retrieval. LlamaIndex Workflows: a step or retriever abstraction. Limit: filtering logic is duplicated per workflow and can diverge between teams.
- Tool-call authorization: LangGraph: tool nodes and routing edges. LlamaIndex Workflows: tool definitions and steps. Limit: authorization written into tool code or prompts depends on every author getting it right.
- Persisted state and checkpoints: LangGraph: checkpointers and memory stores. LlamaIndex Workflows: context store and serialization. Limit: whatever reaches state is stored with it, so PHI needs handling before it is persisted.
- Human approval: LangGraph: interrupts. LlamaIndex Workflows: events and steps that wait for input. Limit: each workflow defines its own approval logic, so escalation rules vary.
- Audit evidence: Both: application-level records the team chooses to emit. Limit: evidence shape differs by framework and workflow, which complicates mapping to 164.312(b).
Persisted state is part of the PHI footprint
If PHI reaches a checkpoint, a memory store or serialized workflow context, that store holds ePHI of a regulated entity. The Security Rule applies to it the same way it applies to a database. Where the store lives, who operates it and who can read it are deployment decisions, and they belong in the organization's risk analysis.
The practical consequence is ordering. A check that redacts PHI after state is saved protects nothing in the saved copy. Enforcement needs to act on content before it enters model context, tool arguments or persisted state, and the evidence needs to show that it did.
Hosted components raise a separate question. HHS guidance says covered entities may disclose PHI to a business associate with satisfactory assurances, in the form of a business associate agreement (BAA), and that BAAs flow down to subcontractors. The same guidance on cloud providers says a BAA is required, and that HIPAA sets no requirements specific to processing outside the United States while noting that risk varies by location. Model providers, state stores and any enforcement service in the path each need to be accounted for.
Human approval and consistency across workflows
Both frameworks let a workflow pause for a person to review an action. That solves escalation for a single workflow. It does not by itself give a security team one escalation policy across fifty workflows built by different teams. Thresholds, approver roles and the definition of a sensitive tool call end up encoded separately, and changing a rule means changing every workflow that implements it.
The same applies to PHI redaction and access limits. When rules live in graph nodes, steps and prompts, drift is the default outcome, and a bypass can come from a new code path, a prompt change or a tool added without a check. The framework is not at fault. Orchestration code is simply the wrong place to hold the only copy of a security policy.
Enforcing policy in the execution path
The alternative is a layer in the execution path that sees each model call, retrieval result and tool call regardless of which framework produced it. Trussed works this way. It enforces policy at runtime, not reviewed after the fact, with the actions allow, redact, block, rate limit and escalate for human approval. Its scope is individual interactions and tool calls, across models, agents, tools and MCP servers.
Because enforcement happens at the point of each interaction, it also produces the evidence: a record of what policy applied and what was done. That audit-ready evidence is mapped to HIPAA, including 164.312(b) audit controls, in the same form for LangGraph and LlamaIndex Workflows. Trussed deploys self-managed in the customer cloud, with configurable retention and data residency, and runs in production with healthcare and pharma organizations.
This does not replace framework-level logic. Nodes, steps and interrupts still carry business flow. The runtime layer carries the security policy, so it stays consistent when workflows change.
Five HIPAA control points in an agent workflow
PHI in prompts and retrieved context
Model calls and retrieval results can carry PHI into and out of the model.
Tool-call authorization
Each tool call can read or write ePHI in another system.
Persisted state and checkpoints
Saved workflow state can hold ePHI at rest.
Human approval
Escalation rules need to apply the same way in every workflow.
Audit evidence
Controls only help if you can show what was allowed and what happened.
Map your enforcement points before the next audit
Walk through your LangGraph and LlamaIndex Workflows deployments with an expert and see where runtime enforcement and audit-ready evidence fit.
Explore Agent and Tool-Call Enforcement