Running LlamaIndex Evaluations on PHI Without Sending Data to a Non-BAA Judge Model
Treat every LLM-as-judge call as a PHI egress point. Send judge calls only to an endpoint covered by a BAA or to a model you host inside your own boundary, or remove PHI before the call leaves, and set the judge model explicitly instead of relying on a default. Then keep records showing which destination each call went to and which policy decided it.
Where PHI flows in a LlamaIndex evaluation pipeline
LLM-as-judge evaluators such as faithfulness, relevancy and correctness work by sending text to a second model that scores it. The payload typically includes the user query, the retrieved context chunks and the generated response, and a correctness check may also include a reference answer. In a RAG application over clinical notes, claims or patient messages, every one of those fields can contain PHI. The judge call is therefore an egress point, in the same way production inference is.
Batch evaluation runs multiply the exposure. A single run can send thousands of payloads to the judge in minutes, so one misconfigured default can move a large volume of PHI before anyone notices. The data around the judge call also matters. Evaluation datasets, stored scores and any retained inputs and outputs are ePHI that your organization creates, receives, uses or maintains, and the Security Rule applies to them.
The regulatory question is straightforward. HHS states that PHI may be disclosed to a business associate when the covered entity has satisfactory assurances in a written contract or arrangement, a business associate agreement. HHS also states that covered entity and business associate customers must obtain a BAA with a cloud service provider. A judge endpoint run by a provider with no BAA, receiving payloads that contain PHI, should be treated as a disclosure that has no such assurance behind it. De-identified data under 45 CFR 164.514 is the other route, and it carries its own conditions.
Choosing a judge routing pattern
There are three workable patterns and one fallback. They differ in how much operational work they add and in how much evaluation quality they may cost. The fidelity points below are engineering judgment to be tested on your own data, not measured results.
- BAA-covered hosted judge: Judge calls go to a hosted endpoint under a signed BAA that covers this workload. Least engineering effort and the strongest judge models are available. Limits: the BAA scope must be confirmed against the actual contract and endpoint, and someone must own the inventory of which endpoints are covered.
- Self-hosted judge: An open-weight or otherwise self-managed model runs inside your boundary, so payloads never reach a third-party provider. Limits: you carry hosting and capacity work, the judge may score less reliably than a larger hosted model, and stored evaluation data stays in scope for the Security Rule.
- De-identify or redact before the judge: PHI is removed from the payload before it leaves, which allows a wider choice of judge. Limits: redaction can strip details a faithfulness or correctness judge needs, which may lower fidelity. A pipeline redaction step is not de-identification under 164.514 unless the method is shown to meet that standard, and residual PHI is possible.
- Block when no compliant route exists: If a call targets a non-covered endpoint with PHI present, it is refused and the run reports a failure. This costs completeness of results but keeps the constraint intact.
Configuring evaluator calls to follow the chosen pattern
The aim is that no evaluator can reach a judge you have not approved. Work through the steps below in order, and repeat them whenever evaluators or evaluation datasets change.
- 1. Inventory evaluators and judge configuration: List every evaluator and batch runner in your repositories and notebooks, and find every place a judge model is set, whether as an application-wide default or a per-evaluator argument. A default that resolves to a hosted model with no BAA is the failure to design against.
- 2. Classify evaluation data: Label each dataset as containing PHI, de-identified, or synthetic. The label decides which judge routes are permitted for it.
- 3. Set the judge explicitly: Construct each evaluator with a named judge model that points at an approved endpoint. Avoid implicit defaults so a missing setting fails loudly instead of falling back to another provider.
- 4. Route by data class: PHI-bearing datasets go only to a BAA-covered or self-hosted judge. Redacted datasets may use a wider set. Synthetic data can use any approved judge.
- 5. Test redaction on realistic payloads: Run your redaction step on representative queries, contexts and responses, review what remains, and compare judge scores before and after. Do not assume that no PHI survives.
- 6. Keep artifacts inside the boundary: Store datasets, scores and run outputs in the same controlled environment as the source data, with defined retention.
Enforcing the constraint at runtime
Configuration alone relies on every engineer getting every evaluator right. A second layer places a policy check on the judge call itself, so the decision does not depend on how a given notebook was written. The check inspects the call, then allows it to a covered or self-hosted destination, redacts the payload, or blocks it.
Trussed enforces policy in the execution path on model calls, including judge model calls. A call that would send PHI to a non-BAA model can be redacted or blocked at runtime, not reviewed after the fact, and the decision is recorded as audit-ready evidence. Trussed can be deployed self-managed in the customer cloud, with configurable retention and data residency, and it runs in production with healthcare and pharma organizations. This layer does not replace the BAA inventory or the redaction testing above. It enforces the routing you have already decided.
Evidence that the constraint held
An auditor will want to see that evaluation traffic carrying PHI never reached a non-BAA endpoint, and a statement of intent is not enough. The audit controls at 45 CFR 164.312(b) sit within the Security Rule technical safeguards, and the records below are a practical set to retain against them. They are proposals for your team, not a prescribed list, so agree the mapping with your compliance and legal teams.
Start with the routing policy, versioned, showing which destinations are permitted for each data class. Add the BAA inventory, with the endpoint, the scope of coverage and the contract owner for each. Then keep per-call decision records: timestamp, evaluator or run identifier, destination, the allow, redact or block outcome and the policy version that produced it. Blocked calls matter as much as allowed ones, because they show the control working. Finally, retain dataset classifications and redaction test results, including what residual PHI was found and how it was handled.
Together these let you answer the question in a form that can be checked: for a given run, which judge received which data, under which agreement, and what the policy decided. Trussed generates this evidence from enforcement and maps it to HIPAA, including 164.312(b) audit controls. If you enforce routing another way, the same record types apply.
PHI path in an evaluation run
Evaluation inputs
Query, retrieved context and generated response from the RAG application.
Judge call
The point where text leaves the pipeline for a scoring model.
Route decision
BAA-covered endpoint, self-hosted judge, redact first, or block.
Evidence
Per-call decision records, routing policy and BAA inventory.
Put policy on the judge call
See how allow, redact and block decisions on model calls produce audit-ready evidence for a regulated enterprise.
Request a Demo