Healthcare AI incident response

    Conducting a HIPAA Breach Risk Assessment After an AI Agent Over-Retrieves PHI Through an MCP Server

    Treat the over-retrieval as a potential impermissible use or disclosure, then establish what the agent retrieved and where the data went. Under 45 CFR 164.402, an acquisition, access, use, or disclosure of PHI not permitted under Subpart E is presumed to be a breach unless you demonstrate a low probability of compromise through a documented risk assessment. That demonstration depends on interaction-level records of tool calls and returned data. This guide is operational and is not legal advice.

    Scope the event and reconstruct the data path

    1. 1

      1. Preserve the record

      Secure tool call records, returned payloads, session context, and any transcripts before retention schedules remove them.

    2. 2

      2. Define the permitted scope

      Document the task, the requesting user or process, and the PHI that task legitimately required under your minimum necessary determinations.

    3. 3

      3. Reconstruct the call

      Capture the tool invoked, the parameters sent, the identity and credentials used, and the records actually returned.

    4. 4

      4. Map the data path

      Trace the payload from the source system through the MCP server and agent context to any model provider, downstream tool, or storage location.

    5. 5

      5. Test the boundary

      Using your asset inventory and logical boundary, determine whether the ePHI was transmitted to external third parties such as cloud service providers.

    6. 6

      6. Count what was exposed

      Identify the individuals and data elements in the over-retrieved set, separate from the data the task required.

    Evidence to hold before you conclude

    • Tool call records with parameters, timestamps, and the invoking user and agent identity
    • The records or fields returned for each call, not only that a call occurred
    • The credentials or scope in effect when the call ran
    • Policy decisions applied to the call and any approvals recorded
    • Context and transcript retention details, including where copies persist
    • Architecture documentation showing whether payloads or context can leave your boundary

    Start with the legal question, not the technology

    Under 45 CFR 164.402, a breach is the acquisition, access, use, or disclosure of PHI in a manner not permitted under Subpart E that compromises the security or privacy of the PHI. The presumption of breach applies only to events that were not permitted. So the first question is whether the over-retrieval was an impermissible use or disclosure at all, or whether an exclusion in paragraph (1) of the definition applies. Whether data entering an agent's context, a model provider, or a downstream tool counts as an impermissible use or disclosure is a determination for your privacy officer and counsel. Retrieving more than a task required does not settle that question on its own.

    If the event is not permitted, the entity must demonstrate a low probability that the PHI has been compromised, based on a risk assessment of at least the required factors. HHS also states that covered entities and business associates have discretion to notify without first performing a risk assessment. Decide early which route you are taking, because it determines what evidence you need to assemble and how quickly.

    The 2013 rule replaced the earlier significant-risk-of-harm standard with the low-probability-of-compromise standard. If your incident templates still use pre-2013 wording, update them before using them for an agent event. This reflects the rule as of November 2026.

    Map the facts to the four risk assessment factors

    Once the data path is established, apply the four factors to the documented facts. Each conclusion should point to a specific record, not to an assumption about how the agent or server is expected to behave. Weigh the factors together, because the question is the overall probability of compromise.

    • Nature and extent of the PHI: List the data types and identifiers returned, any clinical or financial detail, and the number of individuals. Evidence: the returned payload and the retrieval parameters.
    • The unauthorized person who used or received the PHI: Identify every recipient: the requesting workforce member, the agent runtime, a model provider, or a downstream tool. Note whether each is bound by obligations to protect the data. Evidence: the data-path map and the agreements covering each party.
    • Whether the PHI was actually acquired or viewed: Distinguish data matched on the server from data returned into context, and data that a person or system actually read or processed. Evidence: response records and context contents.
    • Extent of mitigation: Record what was done to reduce risk, such as removing the data from context and storage, obtaining written confirmation from recipients, narrowing credentials, and disabling the tool. Evidence: dated actions and recipient attestations.

    Decide on notification and document either way

    The burden sits with the entity. If you conclude notification is not required, you must be able to demonstrate that the event did not constitute a breach as defined in 164.402. HHS states that the entity should keep documentation that all required notifications were made, or documentation that notification was not required. The latter is either a risk assessment showing a low probability of compromise or the application of another exception to the definition of breach.

    If the facts show the PHI left your boundary, or you cannot establish where it went, the evidence gap itself weighs against a low-probability conclusion. In that situation, the discretion to notify without completing an assessment is a legitimate option. Where notification is required, the general rule requires notice to each individual whose unsecured PHI has been, or is reasonably believed to have been, accessed, acquired, used, or disclosed as a result of the breach. Notice content and timelines should be confirmed with counsel against the current rule.

    A complete record includes the incident timeline, the scoping results, the data-path map, the analysis of each factor with the supporting records, the mitigation actions, the decision, and the names of the approvers. Feed the findings back into your Security Rule risk analysis, and confirm that your asset inventory and logical boundary include agent tool access.

    Reduce recurrence with runtime controls at the tool call

    Most of the difficulty in this assessment comes from reconstructing events after the fact. Controls applied at the MCP tool call change that. Four policy actions apply to over-retrieval. Allow permits a call that fits the task. Redact removes fields beyond what the task needs before the payload reaches the agent. Block stops a call that falls outside the permitted scope. Escalate routes a call to a human for approval before data is returned. Each decision, made against a defined policy, is also a record of how minimum necessary was applied to that task.

    Trussed enforces policy in the execution path of agents, tools, and MCP servers, applying allow, redact, block, rate limit, and escalate for human approval to individual interactions and tool calls. Enforcement is at runtime, not reviewed after the fact, and it generates audit-ready evidence mapped to HIPAA. It can be self-managed in the customer cloud with configurable retention and data residency. Trussed runs in production with healthcare organizations. It supplies runtime enforcement and evidence. It is not an incident response, GRC, or observability tool, so the assessment, the determination, and the notification decision remain with your privacy and security teams.

    Assessment path for an MCP over-retrieval event

    Scope

    Identify the task, the tool call, and the PHI returned.

    Trace

    Map where the data traveled after it left the source system.

    Assess

    Apply the four risk factors to the documented facts.

    Decide and record

    Notify or document why notification is not required.

    Strengthen the evidence behind your next determination

    See how policy enforcement on MCP tool calls limits PHI over-retrieval and produces audit-ready evidence for HIPAA assessments.

    Explore Agent and Tool-Call Enforcement