Implementation Guide | Healthcare

    Capping LlamaIndex Top-K and Chunk Scope to Meet HIPAA Minimum Necessary

    Set similarity_top_k, chunk size and overlap, metadata filters, similarity cutoffs and node postprocessors per documented role and purpose, not as one global default. These settings cap how much PHI can reach the model, but they are static application configuration. They do not show what was retrieved and sent in each interaction. Enforcing allow, redact or block on retrieved context at runtime, and keeping evidence per interaction, closes that gap.

    1. Document the minimum necessary scope for each role and purpose, and name an approver.
    2. Classify each RAG use case as routine and recurring or non-routine, and record any exemption relied on.
    3. Set top-k, chunk size and overlap, filters and cutoffs per use case, with a written rationale.
    4. Tag documents at ingestion and derive filters from authenticated role, not user input.
    5. Decide what must be checked at runtime on every interaction, and where that check sits in the execution path.

    What Minimum Necessary requires of a retrieval pipeline

    Under 45 CFR 164.502(b), a covered entity or business associate must make reasonable efforts to limit PHI to the minimum necessary to accomplish the intended purpose of a use, disclosure or request. Section 164.514(d) sets out the supporting requirements. For disclosures made on a routine and recurring basis, the entity must implement policies and procedures, which may be standard protocols, that limit PHI to the amount reasonably necessary. For non-routine requests and disclosures, criteria must be developed and applied individually. Business associates are named directly in 164.502(b), so a vendor handling PHI in a RAG pipeline may carry the same obligation. This is current text as of November 2026.

    The rule contains no numeric limit and names no retrieval parameter. Terms such as reasonable efforts, amount reasonably necessary and standard protocols leave the mapping to engineering judgement. Reading fixed, documented retrieval configurations as standard protocols for recurring query types is an interpretation, and it should be recorded as one. Two further points matter for design. HHS states the rule does not prohibit using an entire medical record where documented policies and procedures identify the whole record as reasonably necessary for a stated purpose. And some uses and disclosures are outside the standard, including treatment disclosures between providers, disclosures to the individual, and disclosures the individual has authorized. Confirm and document any exemption before relying on it. Whether text placed in model context counts as a use or a disclosure in your deployment is a question for counsel.

    Retrieval settings that determine how much PHI reaches the model

    Each of the following settings bounds the PHI that can enter model context. Treat them as a set, since a tight top-k does little if each chunk carries an entire note. Confirm the exact behaviour against the LlamaIndex version you deploy.

    • similarity_top_k: Sets the maximum number of nodes the retriever returns per query. It is the upper bound on how many chunks can enter context, so it should be chosen per use case and not left at a shared default.
    • Chunk size and chunk overlap: Set at ingestion by the node parser. Larger chunks carry more surrounding text with each relevant passage. Overlap repeats text across adjacent chunks, so the same PHI can appear more than once in a single response.
    • Metadata filters: Restrict candidate nodes by attributes such as facility, document type, department or patient. They are only as reliable as the tagging applied at ingestion.
    • Similarity cutoff: Drops nodes below a relevance score so weak matches do not fill the top-k. The score measures relevance to the query, not whether the requester is entitled to the content.
    • Node postprocessors: Run after retrieval and before synthesis. They can filter, rerank or transform nodes, which makes them a place to remove or mask fields the purpose does not need.

    Defining scope by role and purpose

    Because the standard is tied to the intended purpose, a single global retrieval setting is hard to defend. Start by classifying each RAG use case as routine and recurring or non-routine. A recurring use, such as a coding-support assistant answering from a defined set of document types, can run on a fixed protocol: a documented top-k, chunk size, filter set and cutoff, approved by a named owner. A non-routine use needs criteria applied to the individual request, which in practice means narrower defaults plus a path for human review or escalation.

    Then express the scope as data. Tag documents at ingestion with the attributes your role definitions need, and derive filters from the authenticated user's role and the stated purpose, not from values the user can type into a prompt. Record the rationale for each limit, since reasonable efforts is the standard you will be asked to demonstrate. Expect tradeoffs: a lower top-k and smaller chunks reduce exposure but can reduce answer completeness, so tune per use case and document why the chosen point is reasonable.

    Where application configuration stops being enough

    All of the settings above live in application code, are set once, and apply to whatever path executes. That creates several gaps. A value can drift between environments or be overridden in a code path nobody reviewed. A filter built from mis-tagged metadata will pass content it should not. A similarity cutoff cannot tell an authorised passage from an unauthorised one. A postprocessor that redacts only works if every query path includes it.

    The larger gap is proof. A configured limit shows intent, not behaviour. Application logs can be used to reconstruct what happened, but reconstruction after the fact depends on what the application chose to record. An auditor asking which chunks and which PHI were sent to the model for a given interaction needs a record made at the point of enforcement. Configuration remains necessary, since it sets the intended scope. What it cannot do is verify, per interaction, that the scope was applied.

    Enforcing and evidencing scope at runtime

    Trussed addresses the runtime half of this problem. It enforces policy in the execution path, applying allow, redact or block to retrieved context for each interaction before it reaches the model. Enforcement happens at runtime, not in review after the fact, so the configured scope is checked against what is actually about to be sent.

    The same enforcement produces audit-ready evidence of what was permitted, mapped to HIPAA, including 164.312(b) audit controls. That evidence is generated by the enforcement step itself, not assembled later from application logs. Trussed runs in production with healthcare and pharma organizations, and can be deployed self-managed in the customer cloud with configurable retention and data residency. It does not replace the role and purpose definitions described above, or the legal determination of how Minimum Necessary applies to your deployment. Those remain your decisions. Enforcement and evidence make them testable.

    From Minimum Necessary to enforced retrieval scope

    Define scope

    Document what PHI is reasonably necessary for each role and purpose.

    Configure retrieval

    Apply top-k, chunking, filters, cutoffs and postprocessors to match.

    Enforce at runtime

    Apply allow, redact or block to retrieved context in the execution path.

    Retain evidence

    Keep a per-interaction record of what was permitted and sent to the model.

    Show what your retrieval limits actually permitted

    Walk through how policy enforcement on retrieved context and per-interaction evidence fit your LlamaIndex deployment.

    See How Evidence Is Produced
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.