What counts as evidence of individualized consideration
Evidence of individualized consideration is a decision-level record, not a statement of policy. For one AI-assisted determination, the record should show which member-specific inputs the system retrieved, how they shaped the recommendation, which policy rules applied, and where a human reviewer confirmed or changed the outcome.
Audit artifacts to specify for each determination
Specify these four artifacts for every AI-assisted determination. Together they form the evidence chain for one decision.
| Artifact | What the record shows |
|---|---|
| Member inputs accessed | Which records the AI retrieved for this member, and when. |
| Policy applied | Which rules and criteria version governed the decision. |
| AI output | The recommendation and the inputs that influenced it. |
| Human review | Who reviewed it, what they saw, and what they decided. |
Runtime controls that generate the evidence
These runtime controls produce the artifacts above as a by-product of normal operation. Approval workflows, the fifth control, are covered under human review below.
- Agent identity and permissionsEach AI component acts under a distinct identity with least-privilege access, so records show who or what retrieved member data and that access was scoped to the case.
- Tool-call loggingEvery retrieval of member records is logged with the member or case reference, the data category requested, the caller, and the time.
- Runtime policy enforcementRules are evaluated at the time of the call, and the outcome (allowed, blocked, escalated) is recorded with the policy version.
- Decision correlationA single decision identifier links retrievals, model output, policy checks, and reviewer actions across systems.
Human review checkpoints that hold up under audit
Approval workflows
Defined actions route to a human before proceeding, and the routing and response are captured in the same record.
What the review record should contain
The human review artifact shows who reviewed the determination, what they saw, and what they decided. Because a single decision identifier links reviewer actions to retrievals, model output, and policy checks, the plan can show where a reviewer confirmed or changed the outcome.
Where current logging and review processes fall short
Gaps appear when a plan cannot produce the full case record for a determination on demand. The checks below help find them.
Evaluating whether workflows can produce reproducible records
Test reproducibility directly. Select a sample of past determinations and ask each system owner and vendor to produce the full case record from identifiers alone. Then check:
- Whether the records agree with one another.
- Whether versions can be identified.
- Whether the same inputs reproduce the same output, or whether differences can be explained.
Useful evaluation questions include:
- Does the vendor expose member-level data access logs, or only aggregate metrics?
- Can logs be exported in a form your team can join across systems?
- Are records immutable or tamper-evident, and for how long are they retained?
- Can the plan, rather than only the vendor, retr