Model risk has become a major focus area as financial institutions rely increasingly on quantitative models for critical decisions. Understanding model risk management is essential for FRM candidates.

What Is Model Risk?

Model risk is the risk of adverse consequences from decisions based on incorrect or misused models. It can manifest through:

  1. Model errors: Incorrect assumptions, flawed mathematics, or coding bugs
  2. Model misuse: Using a model outside its intended scope or limitations
  3. Implementation errors: Incorrect data inputs or technical implementation mistakes

SR 11-7: Fed Guidance on Model Risk Management

The Federal Reserve's SR Letter 11-7 (2011) is the foundational document:

Key Principles

  1. Model development should be based on sound theory and empirical evidence
  2. Rigorous model validation must be performed by a team independent from developers
  3. Ongoing monitoring tracks model performance over time
  4. Model inventory maintains a comprehensive catalog of all models
  5. Board and senior management are responsible for effective model risk governance

Model Lifecycle

  1. Development → 2. Documentation → 3. Validation → 4. Approval → 5. Implementation → 6. Monitoring → 7. Annual Review → 8. Retirement

Model Validation

Conceptual Soundness

  • Are the underlying assumptions reasonable?
  • Is the mathematical framework appropriate?
  • Are limitations well understood?

Outcomes Analysis

  • Does the model produce accurate results?
  • Backtesting: Compare predictions to actual outcomes
  • Benchmarking: Compare against alternative models

Sensitivity Analysis

  • How sensitive are outputs to changes in inputs?
  • Are there cliff effects or instabilities?
  • How does the model perform under stress?

Emerging Model Risk Challenges

AI/ML Models

Machine learning creates new model risk dimensions:

  • Black-box problem: Difficulty explaining model decisions
  • Overfitting risk: Models that perform well historically but fail on new data
  • Data bias: Training data reflecting historical biases
  • Concept drift: Model degradation as relationships change
  • Regulatory acceptance: Can regulators validate complex ML models?

Climate Models

New type of model risk with unique challenges:

  • Very long time horizons (decades)
  • Limited historical data
  • High parameter uncertainty
  • Novel interconnections not captured by existing models

Governance Framework

Three Lines of Defense for Models

  1. Model developers/users: Initial testing, monitoring, appropriate use
  2. Model validation team: Independent validation, challenge, approval
  3. Internal audit: Assurance over the model risk framework

Model Risk Reporting

  • Model inventory completeness
  • Validation status and findings
  • Model performance metrics
  • Exceptions and limitations
  • Risk ratings and materiality assessments

Model risk appears across multiple FRM topics — practice with our comprehensive question bank!