Model Validation

A model can look impressive to the team that built it and still be wrong. Its assumptions may not hold, its training data may not represent the people it will be used on, its accuracy may have been measured on data it had already seen, or the code that runs it in production may differ from the version that was tested. Model validation is the step that catches these problems. It is a second, independent look before a model is trusted. Someone other than the developers examines how the model was designed, tests it independently, checks its implementation and decides whether it is fit for its intended purpose, with conditions or limits where needed.

Validation is a long-established practice in banking, where supervisory guidance on model risk management has emphasized independent validation and effective challenge for years. It applies equally to AI and machine learning models, although complex models and generative AI require additional techniques, such as fairness testing, robustness testing and evaluation of open-ended outputs. For data science, risk, audit and compliance teams, validation is the main way to gain confidence that a model can be trusted before it influences real decisions.

Definition of Model Validation

Model validation is the set of processes and activities intended to verify that a model is performing as expected, in line with its design objectives and business uses, including evaluation of its conceptual soundness, testing of its performance and stability, review of its implementation and data, and identification of its limitations.

It is a core element of model risk management, a required step in AI model governance and was a central expectation of SR 11-7.

Components of Model Validation

Conceptual soundness reviews whether the model's design, assumptions, methods and choice of data are appropriate for its purpose. A model can be technically correct but conceptually wrong for the problem it is meant to solve.

Data review checks the quality, relevance and representativeness of training and testing data, including gaps for particular groups or conditions. Data issues are among the most common validation findings.

Performance testing measures accuracy, stability and robustness on data the model has not seen, including out-of-time samples and stress scenarios. Stress scenarios show how the model behaves in conditions it rarely saw in training.

Fairness and explainability testing check for unjustified differences across groups and whether the model's behavior can be understood. These checks are essential for models that affect people.

Implementation review confirms that the production version matches the validated version and handles inputs correctly. Differences between test and production environments are a frequent source of errors.

Ongoing validation includes monitoring and outcomes analysis after deployment, comparing predictions with actual results.

Validating Machine Learning and Generative AI

Machine learning models need careful checks for overfitting, data leakage and reliance on proxies for protected characteristics. These problems can make a model look far better in testing than it is in practice.

Complex models may require explanation techniques to assess whether they rely on sensible factors.

Generative AI models are validated through evaluation sets, automated scoring, human review, red teaming and checks for risks such as false or unsafe outputs, often using methods described under AI evaluation.

Third-party models need validation of their performance for the organization's specific use, even when vendors provide their own test results.

Importance of Model Validation in AI Governance

Validation catches errors before they matter. Independent testing finds flawed assumptions, data problems and implementation mistakes that developers may miss. Developers are naturally close to their own work and may overlook its weaknesses.

It provides effective challenge. Validators with expertise and independence can question design choices and push for improvements. Their challenge is only effective if they have the standing to require changes.

It also builds confidence. Leaders, regulators and customers can rely on models that have been independently validated. Validation reports also give model users clear guidance on limits.

Model Validation and Financial Management

Flawed models can lead to large losses, so validation is a cost-effective control for material models.

Validation requires skilled staff or external experts, so effort should be scaled to each model's risk tier.

Findings that lead to simpler, more robust models can reduce long-term maintenance and monitoring costs.

Clear validation records reduce the cost and time of audits and regulatory reviews.

Common Challenges

Validation teams can become bottlenecks as the number of models grows. Risk tiering and automated testing help focus validators on the models that matter most.

Validators may lack the specialized skills needed for advanced AI and generative models. Training, hiring and external experts can help close the gap.

Validation done only before launch can miss problems that emerge later, so ongoing validation is needed.