How to Test an Insurance AI Model for Proxy Discrimination Without Race Data

What proxy discrimination testing means in insurance AI

Proxy discrimination testing evaluates whether an underwriting or pricing AI model may create different outcomes for protected groups even when the model does not directly use protected class data. In insurance, this issue is especially important because many inputs can be facially neutral while still being correlated with race or ethnicity.

When direct race data is not collected, testing can still be performed by using a documented proxy estimation method. The most common approach described in the supplied content is Bayesian Improved Surname Geocoding, often referred to as BISG. BISG estimates race or ethnicity probabilities from surname and geography. These estimates are then used for validation and disparate impact analysis, not for production underwriting decisions.

Core methods for testing without direct race data

A practical testing approach starts by estimating protected class probabilities outside the production scoring path. Those probabilities can then be used to compare model outcomes across estimated demographic subgroups. The analysis should include adverse impact ratio comparisons and should also examine whether model inputs, intermediate scores, or final outputs are correlated with protected class estimates.

Final outcome testing is necessary, but it is incomplete on its own. A governance team should also test whether input variables or intermediate scores are correlated with estimated protected class probabilities, because facially neutral features may act as proxies.

A repeatable implementation workflow

The testing process should be designed as a controlled validation workflow rather than an informal analysis. A practical workflow estimates protected class probabilities, compares model outcomes across estimated demographic subgroups, tests feature-level proxy relationships, and records the results in a form that can be reproduced during governance review or regulatory examination.

• Use a recognized proxy estimation method to generate probabilistic protected class estimates outside the production scoring path.
• Compare model outcomes across estimated demographic subgroups using documented disparate impact methods.
• Supplement final outcome testing with feature-level proxy analysis.
• Version the method, thresholds, data vintages, model versions, results, and remediation actions.
• Keep testing repeatable, separated from production scoring, tied to model versions, and documented for regulatory examination.

Architecture controls for defensible testing

The testing architecture should keep proxy estimation and production decisioning separate. BISG requires reference data such as surname and geography files, but those inputs should be introduced into a validation pipeline, not into the live underwriting model. The purpose is to test for possible discriminatory effects, not to enrich production scoring data with inferred protected class attributes.

A practical setup links each testing run to a specific model version, input dataset, outcome dataset, and methodology configuration. The configuration should include geography granularity, surname and geography data vintage, probability thresholds, subgroup definitions, weighting rules, and adverse impact calculations. Without this metadata, the insurer may be unable to reproduce a prior result during an examination or explain why two testing runs produced different conclusions.

Runtime monitoring is less standardized than pre-deployment testing, but the governance direction is clear: testing should not be a one-time event. Once deployed, underwriting and pricing models should be periodically retested using the same controlled method, especially when the model is changed, input data shifts, or business rules around the model are updated. Runtime governance and audit logging can support this by preserving records of model versions, approvals, policy decisions, and monitoring events associated with deployed AI systems.

Governance decisions that determine whether the test is defensible

The reliability of proxy discrimination testing depends on consistent governance choices. Specific mandated thresholds were not confirmed in the provided evidence. Insurers should document their chosen metrics, thresholds, escalation rules, and jurisdictional rationale rather than assuming a universal standard.

Because BISG produces probabilistic estimates, the process should not treat those estimates as definitive protected class labels. The method should be used to support testing and governance, not as a production underwriting input. The most defensible approach is to preserve the complete context for each run, including the model version, the data used, the assumptions applied, and the resulting analysis.

Common implementation questions