LLM Evaluation Metrics Explained: Groundedness, Toxicity, and Refusal Rates
Groundedness, toxicity, and refusal rate are LLM evaluation metrics that help enterprises assess whether model outputs are supported by supplied evidence, safe under content policies, and appropriately constrained. They are useful governance signals, but none is sufficient on its own. Each metric depends on test set design, policy definitions, scoring method, human review, and runtime context.
Evaluation metric overview
What these LLM evaluation metrics actually measure
For governed LLM deployment, these metrics are best understood as three metric families. Each one answers a different operational question, and each one needs to be interpreted with the surrounding application context.
Groundedness
Checks whether generated claims are supported by supplied or retrieved context.
Toxicity
Estimates whether output may contain harmful, abusive, hateful, or otherwise unsafe content.
Refusal rate
Measures how often the system declines to answer, segmented by prompt category and policy intent.
How to interpret groundedness, toxicity, and refusal rate
These metrics are useful governance signals, but none should be treated as a complete answer on its own. The result depends on the test set, policy definition, scoring method, review process, and runtime context. A single aggregate score can obscure meaningful differences across prompt types, user roles, regions, languages, and interaction patterns.
| Metric | What it supports | Important limitation |
|---|---|---|
| Groundedness | Evidence support, citation review, answer blocking, or routing to human review when an output is not supported by supplied context. | Groundedness is not the same as factual accuracy. If the context is wrong, incomplete, outdated, or unauthorized, a grounded answer can still be unsuitable for enterprise use. |
| Toxicity | Content safety review, moderation routing, incident analysis, and policy threshold monitoring. | Toxicity scores can support compliance workflows, but they are not compliance proof by themselves. Scores should be mapped to policy thresholds, reviewed in context, and stored with audit records and remediation decisions. |
| Refusal rate | Policy adherence, usability review, detection of excessive caution, and detection of unsafe compliance. | There is no universal good rate. The target depends on prompt category. Harmful prompts should generally show appropriate refusal, while benign business prompts should not be blocked unnecessarily. |
Where these metrics fit in enterprise runtime governance
Offline evaluation, pre-release testing, and runtime monitoring serve different purposes. Offline evaluation compares candidate models or configurations under controlled conditions. Pre-release testing validates the full application, including prompts, retrieval, policies, tools, and user flows. Runtime monitoring observes production behavior, where users may introduce unexpected inputs and agents may take downstream actions.
For ordinary chat applications, groundedness and toxicity are often evaluated on the final answer. For AI agents, that is not enough. Governance teams should also evaluate intermediate steps such as tool calls, API parameters, delegated tasks, permission use, and handoffs between agents. A final response may appear safe while an agent has already attempted an excessive or unauthorized action.
Runtime monitoring can use these metrics as control signals. A groundedness failure may trigger human review, citation suppression, or response blocking. A toxicity alert may route the exchange to moderation or incident review. A refusal anomaly may indicate policy regression, prompt injection, excessive caution, or unsafe compliance. These decisions generally require more than the score itself. Useful evidence includes user role, policy version, prompt, context, output, evaluator result, downstream action taken, and remediation decision.
This is where runtime governance becomes operational. Controls should be connected to identity, permissions, least privilege, tool approval workflows, audit logging, and incident response. For agent deployments, metrics should not only describe behavior after the fact. They should inform whether an agent can call a tool, access a data source, complete an action, or require human approval.
Implementation choices that determine metric reliability
Metric reliability is shaped by how the evaluation is designed and operated. The same metric name can produce different governance value depending on the dataset, prompt categories, policy thresholds, evaluator version, review workflow, and production context.
For enterprise use, reliability improves when evaluation results are tied to the decision they support. A groundedness score is more useful when the organization knows whether that score triggers a warning, a release gate, response blocking, citation suppression, or human review. A toxicity score is more useful when it is interpreted against a documented policy threshold. A refusal-rate measurement is more useful when it is segmented by prompt category and policy intent.
Practical governance practices for using LLM safety evaluation metrics
- Map each metric to a decision: Use groundedness for evidence support, toxicity for content safety, and refusal rate for policy adherence and usability. Avoid reporting metrics without the decision they support.
- Segment before interpreting: Break results down by prompt type, use case, user role, language, region, and interaction type. Aggregate scores can hide localized failures.
- Version the full evaluation context: Store model version, prompt version, policy version, dataset version, evaluator version, and retrieval configuration for auditability and reproducibility.
- Combine automation with human review: Use automated scoring for scale, but apply human review for disputed outputs, high-impact decisions, threshold tuning, and incident analysis.
- Tie thresholds to controls: Define pass, warn, and fail thresholds in advance and connect them to release gates, rollback criteria, alerts, human approval, or permission reduction.
Common questions about LLM evaluation metrics
Is groundedness the same as factual accuracy?
No. Groundedness measures whether an output is supported by the supplied context. If the context is wrong, incomplete, outdated, or unauthorized, a grounded answer can still be unsuitable for enterprise use.
Can toxicity scores be used as compliance evidence?
They can support compliance workflows, but they are not compliance proof by themselves. Scores should be mapped to policy thresholds, reviewed in context, and stored with audit records and remediation decisions.
What is a good refusal rate?
There is no universal good rate. The target depends on prompt category. Harmful prompts should generally show appropriate refusal, while benign business prompts should not be blocked unnecessarily.
How should agentic systems be evaluated differently?
Evaluate not only the final response, but also tool calls, permissions, API parameters, delegated actions, and approval steps. Agent behavior can create risk before the final answer is generated.
Connect LLM evaluation metrics to runtime controls
Trussed AI provides runtime governance and security for enterprise AI agents, including policy enforcement, monitoring, permissions, tool approval workflows, and audit logging.
Explore Runtime Governance