How to Identify Users Affected by Bad LLM Responses After an Incident

Treat it as an evidence problem, not a log search

After a policy violation, data exposure, harmful output or faulty tool call, the first instinct is to search logs for the offending prompt or response. That finds examples. It does not establish who else received the same kind of output, and a regulator or auditor will ask for the second thing.

The starting decision is what "affected" means. NIST AI RMF 1.0 defines end users as the individuals or groups that use the system for specific purposes. It defines affected individuals and communities more broadly, as all individuals, groups, communities or organizations directly or indirectly affected by AI systems or by decisions based on their output. An incident may therefore involve two populations: people who received a bad response, and people whose information appeared in it.

Threat-modeling scenarios for LLM applications show why the distinction matters. In a user-prompt-disclosure scenario, a model is induced to reveal other users' emails, names, usernames and queries. The people affected are the ones whose data was disclosed, not only the person who saw the output. Write the definition down before querying anything, because it determines which records you need.

Tracing by attacker is also unreliable. With indirect prompt injection, instructions are hidden and can trigger for other users, which makes tracing them back to whoever injected them very difficult. Scoping by who received which output, using interaction-level records, is the more dependable route.

Framework and regulatory context

NIST AI RMF 1.0 gives the process a structure. GOVERN 4.3 states that organizational practices are in place to enable AI testing, identification of incidents, and information sharing. MEASURE 1.2 covers assessing the effectiveness of existing controls, including reports of errors and potential impacts on affected communities. Neither prescribes a log schema, but both assume you can identify incidents and understand who was affected.

Under the EU AI Act, as of October 2026, high-risk AI systems must technically allow for the automatic recording of events (logs) over the lifetime of the system. Whether a particular LLM use case is high-risk is a legal question that needs review. The Act also provides that a serious-incident report must be made immediately after the provider has established a causal link, or the reasonable likelihood of one, between the AI system and the incident. In any event it must be made no later than 15 days after the provider or, where applicable, the deployer becomes aware of the incident. The required report content includes identification of the natural persons involved in verifying the results, which is one reason to document who validated your list.

Sector rules such as HIPAA, NYDFS and NAIC requirements carry their own notification and record-keeping duties. Confirm those with counsel against your own deployment, and design the evidence so it can answer whichever applies.

Where policy enforcement in the execution path fits

Records reconstructed from application logs, prompt histories and provider records are separate artifacts, each created for a different purpose. Evidence produced at the point where policy is applied avoids stitching them together, because the decision and the record come from the same event.

Trussed is one example of this approach. It enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants, applying allow, redact, block, rate limit or escalate-for-approval decisions to individual interactions and tool calls. The audit-ready evidence is generated by that enforcement, so the policy decision sits alongside the interaction it governed. That evidence is mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST. Trussed can be self-managed in the customer cloud, with configurable retention and data residency, which bears directly on the last two evaluation questions above.

The scope is deliberately specific. Trussed operates at the level of individual interactions and tool calls. It does not provide transaction-level governance across tasks and agents, end-to-end tracing of agentic workflows, or model and dataset lineage, so an incident scoped to those layers needs other evidence. For the question this guide addresses, which users received which responses under which policy decision, interaction-level evidence is the relevant layer.