Implementation Guide | Healthcare

    LangSmith Tracing and PHI: How to Keep LangChain Traces Out of a Non-BAA SaaS

    Treat trace export as a data egress path. Where a trace destination has no business associate agreement, no PHI should reach it. Layer four controls: tracing off by default, input and output masking where tracing is needed, restricted egress or a destination inside your boundary, and policy enforcement in the execution path ahead of export. Then prove it with synthetic PHI tests and configuration regression checks.

    Why trace egress is a HIPAA data-flow question

    HHS guidance states that if a covered entity or business associate uses a cloud service provider to maintain ePHI, for example to process or store it, without entering into a business associate agreement (BAA) with that provider, the covered entity or business associate is in violation of the HIPAA Rules. A BAA is the written arrangement that permits disclosure of PHI to a business associate, and it must contain the elements specified at 45 CFR 164.504(e).

    Whether a trace destination falls under this guidance depends on what is sent to it. That is a fact about your application's data flow, not something the guidance decides for you. If prompts, retrieved context, tool payloads or model responses contain PHI and are exported to a SaaS destination, that destination is receiving PHI on your behalf.

    HHS also notes that a cloud provider meeting the definition of a business associate must comply with the applicable HIPAA provisions whether or not a BAA exists. That does not make a non-BAA destination acceptable. The violation sits with the covered entity or business associate that sent the data. As of November 2026 these positions are current, and the Privacy Rule and the 164.312 technical safeguards remain in force.

    The practical rule is simple. Treat every trace destination as a possible recipient of ePHI, and send none to a destination without a BAA. Because the exposure occurs at disclosure, controls need to act before export, not after the data sits with the vendor.

    How PHI reaches a trace and how tracing gets switched on

    A trace records an application run as a tree of steps, and each step typically records its inputs and outputs so engineers can debug behavior. In a LangChain application those steps commonly include the user prompt, documents retrieved and inserted as context, arguments passed to tools, tool results, and the model response. Each can carry PHI. A clinician's question may name a patient, a retriever may return clinical notes, a tool may query a record system, and a model may summarize all of it. Metadata such as user identifiers, session names and tags can also carry identifying values.

    Configuration is the second risk. Tracing is commonly enabled through environment variables or client configuration rather than application code. A variable set in a shared base image, a CI template, a notebook or a developer workstation that can reach production data can start exporting payloads with no code change and no review. Callbacks attached to individual invocations add another place where tracing can be enabled.

    This is why application-level settings alone are hard to verify. The setting lives in many places, a dependency upgrade or copied environment file can reintroduce it, and a code review will not show an environment variable injected at deploy time. Any design that relies on every team remembering a setting will regress.

    Mitigation options and their trade-offs

    No single option covers every case. The table compares the controls available to most teams, with the limits that decide how they are layered.

    • Disable tracing: Removes the egress path for that environment. Costs debugging visibility, and must be enforced in every environment and image to stay off.
    • Mask or hide inputs and outputs: Keeps trace structure while removing payloads. Depends on coverage of every step, nested calls and metadata. Pattern-based masking can miss PHI in free text.
    • Self-hosted or alternative destination: Keeps trace data inside your boundary. Adds operational load, and you own access control, retention and the BAA position of anything involved.
    • Network egress restriction: Blocks traffic to unapproved hosts regardless of application settings. Coarse: it does not show what was sent to an approved host.
    • Runtime enforcement in the execution path: Applies allow, redact or block to content before downstream components see it. Requires policy design and tuning, and it produces evidence of each decision.

    A layered pattern that keeps PHI away from a non-BAA destination

    Layering matters because each control fails differently. Configuration can drift, masking can miss a field, and egress rules cannot inspect content. Combining them means one failure does not become a disclosure.

    Verification should be designed in from the start. Run tests with synthetic PHI that exercise every step type: prompt, retrieval, tool call, response and metadata. Assert that none of the synthetic values appear in anything exported or in any capture of outbound traffic. Run the same tests in CI so a regression fails the build, and add a check that fails when tracing configuration appears in an image or deployment where it is not approved.

    • Inventory every trace and telemetry destination per application and environment, and record whether a BAA covers it.
    • Set tracing to off in base images and templates, and require a reviewed change to enable it anywhere PHI can appear.
    • Restrict outbound traffic to approved hosts so a stray setting cannot reach an unapproved destination.
    • Where tracing is needed, mask or hide inputs, outputs and metadata at the client, or send traces to a destination inside your boundary.
    • Enforce policy in the execution path so PHI is redacted or blocked before it reaches instrumented steps.
    • Test with synthetic PHI in CI and in production-like environments, and keep the results as records.

    Runtime enforcement and audit controls under 164.312(b)

    The controls above reduce what traces can contain, but they are configuration. Runtime policy enforcement works at a different layer. Trussed enforces policy in the execution path across models, agents, tools and MCP servers, with the actions allow, redact and block applied to individual interactions and tool calls. When PHI is redacted or blocked in the path, enforced at runtime and not reviewed after the fact, the payload that reaches downstream components, including any tracing layer placed after enforcement, is the enforced version. It does not replace a tracing tool for debugging, and it does not trace agentic workflows end to end.

    Enforcement also yields evidence. Each decision produces audit-ready evidence of what was allowed, redacted or blocked, and that evidence is mapped to HIPAA. This can support the audit controls standard at 164.312(b), because the organization can show what data was permitted to leave the environment and what was stopped. It is supporting evidence, not a guarantee of compliance, and it does not replace confirming the BAA position of each destination.

    Trussed supports self-managed deployment in the customer cloud, with configurable retention and data residency, so the evidence can stay within your own boundary. It runs in production with healthcare and pharma organizations.

    Where PHI can enter a trace

    Prompt and context

    User input and retrieved documents injected into the prompt.

    Tool calls

    Arguments sent to tools and the results they return.

    Model response

    Generated output that may restate PHI.

    Trace export

    The point where the payload leaves your environment for a destination.

    Review your trace egress paths

    Talk through where PHI can enter your LangChain traces and how enforcement in the execution path can produce audit-ready evidence.

    Talk to an Expert
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.