RA · Foundations
Learning model risk and validation: a guide to the field
A learning guide to model risk and validation: effective challenge, validation methods, supervisory texts, an order of study and how expertise is shown.
- As of
- 17 November 2026
- Kusoma
- 11 minutes
- Ushahidi
- 10 sources, cited in order
- Level
- Introductory to intermediate
- Study time
- 15 minutes
What you will be able to do
- Define a model and model risk.
- Name the two roots of model risk.
- Describe effective challenge and its three conditions.
- List the components of the model lifecycle.
- Level
- Introductory to intermediate
- Study time
- 15 minutes
- Before you start
- The learning guide to the Faculty of Risk Science and Methods, or equivalent background
01What model risk and validation covers
What model risk and validation covers
Model risk and validation is the Academy discipline of model risk and validation, one of the disciplines of the Faculty of Risk Science and Methods. It covers how organisations find out whether a model is fit for its purpose, and how they manage the risk that it is not.
The Federal Reserve guidance on model risk management, issued on 4 April 2011, uses the term model to refer to a quantitative method, system or approach that applies statistical, economic, financial or mathematical theories, techniques and assumptions to process input data into quantitative estimates 1. The same guidance says that model risk can lead to financial loss, poor business and strategic decision making, or damage to a bank's reputation 1.
The discipline sits beside AI and machine learning for risk analysis, scenario analysis and stress testing and uncertainty and decision making. Its core question is simple to say and hard to answer: is this model good enough for this use?
A short example shows why the question needs care. A model built to rank customers for a marketing campaign may be sound for that task and unsafe for setting a credit limit, because the cost of an error is different and the data may not cover the same people. The model has not changed. The use has. The Federal Reserve guidance makes the same point when it says that a fundamentally sound model may exhibit high model risk if it is misapplied or misused 1. Validation therefore always starts from the stated use.
02Why validation matters
Why validation matters
Models now sit inside many decisions. The Office of the Superintendent of Financial Institutions (OSFI) says that the financial services industry is experiencing a rapid rise in digitalisation and model applications, amplified by the surge in AI and machine learning models. It adds that institutions increasingly rely on models to support or drive decision-making, including in areas that traditionally did not use them 3.
Supervisors treat the subject as a risk discipline in its own right. The PRA states that its SS1/23, published on 17 May 2023, sets out its expectations for banks' management of model risk, and that the desired outcome is a strategic approach to model risk management as a risk discipline in its own right 2. In the euro area, the European Central Bank explains that banks may use internal models to estimate their own funds requirements provided they have prior authorisation from the competent authority 4.
The limits of validation are part of the subject. Oreskes, Shrader-Frechette and Belitz argued in Science in 1994 that verification and validation of numerical models of natural systems is impossible, because natural systems are never closed and model results are always nonunique. They argue that models can be confirmed only partially, and that their predictive value is always open to question 5. A learner should treat validation as building justified confidence, not proving a model true.
03The model lifecycle and its checks
The model lifecycle and its checks
A model has a life, and risk arises at each stage. OSFI defines the model lifecycle as the components that define the life of a model, covering all steps for designing, operating and maintaining it until it is decommissioned. The components are model design, including rationale, data and development, then model review, deployment, monitoring and decommission, and they are not necessarily sequential 3.
The Federal Reserve guidance names ways to check a model. These include ongoing monitoring, with process verification and benchmarking, and outcomes analysis, including back-testing 1. It says a guiding principle is effective challenge of models, which depends on a combination of incentives, competence and influence. Incentives are stronger when challenge is more separate from model development 1. Figure 2 sets out the lifecycle.
Governance sits around the lifecycle. OSFI describes a model risk management framework as a set of policies and procedures that reflects the institution's risk appetite for model risk and defines governance requirements, including roles and responsibilities and processes to identify, assess, manage, monitor and report on model risk 3. The text applies to all federally regulated financial institutions in Canada, to the extent consistent with applicable requirements 3.
Fig. 02The model lifecycle
Figure 2 in words
Five layers stacked from top to bottom: model design, including rationale, data and development; model review, highlighted; model deployment; model monitoring; model decommission. OSFI notes that the components are not necessarily sequential and may vary with the type of model.
04A suggested learning order in four stages
A suggested learning order in four stages
The Academy describes study at four levels: introductory, intermediate, advanced and integrative. The four stages below follow the same arc for model risk and validation. They are a suggestion, and a learner can enter at any stage.
Stage one: what a model is. Start with the definition and the idea of model risk. Read the opening sections of the Federal Reserve guidance 1. Write a one-sentence definition of a model and of model risk in your own words, and list three models you meet in daily life. For each, say what decision it informs and what would happen if it were wrong. The exercise shows quickly that models matter in proportion to the decisions that rest on them.
Stage two: validation methods. Next, study how models are tested. Learn conceptual review, ongoing monitoring, benchmarking and outcomes analysis with back-testing 1. Learn data leakage, with the scikit-learn guidance on common pitfalls 6. Read Oreskes and colleagues on what validation can and cannot show 5.
Stage three: governance and lifecycle. Then study governance. Read the PRA SS1/23 2 and the OSFI Guideline E-23 3. Compare how each describes roles, the lifecycle and the inventory of models.
Stage four: AI and machine learning models. Finally, study models that learn from data. Read the NIST AI Risk Management Framework 7 and the European Commission summary of the AI Act 8. Read the argument for interpretable models in high-stakes uses 9.
The order is a suggestion. A practitioner may begin at stage three and return to stage one later. What matters is that each new idea has the ideas beneath it in place.
Fig. 01Four stages of study in model risk and validation
Figure 1 in words
Four steps rise from left to right. Step one, what a model is: the Federal Reserve guidance definitions. Step two, validation methods: monitoring, outcomes analysis, leakage and the limits of validation. Step three, governance and lifecycle: PRA SS1/23 and OSFI Guideline E-23. Step four, highlighted, AI and machine learning models: the NIST framework, the EU AI Act and interpretable models.
Fig. 03The two roots of model risk
Figure 3 in words
Four stages run from left to right: model, intended use, actual use and decision. Model risk arises in two ways, according to the Federal Reserve guidance: the model may have fundamental errors and produce inaccurate outputs for its design objective, or it may be used incorrectly or outside its design. Both feed the decision.
05Six core ideas every learner should hold
Six core ideas every learner should hold
Six ideas carry most of the weight in this discipline. Each is simple to state and takes practice to use.
Model risk has two roots. Errors in the model, and wrong or inappropriate use of a sound one 1.
Challenge must be independent enough. Effective challenge depends on incentives, competence and influence 1.
Validation builds confidence, not proof. Oreskes and colleagues state that confirmation is inherently partial and that complete confirmation is logically precluded 5.
The whole lifecycle matters. Design, review, deployment, monitoring and decommission are all points of risk 3.
Models are an inventory and a governance matter. A framework defines roles, risk appetite and reporting 3.
AI models raise the stakes. Supervisors and standard setters address them explicitly 3 7 8.
06Canonical references and data to know
Canonical references and data to know
A learner should know a short list of primary texts. Read the originals first and treat summaries as an aid.
For supervisory expectations: the Federal Reserve guidance of 2011 1, PRA SS1/23 2 and OSFI Guideline E-23 3. For the European setting: the ECB page on internal models 4. For AI: the NIST AI RMF 7, the EU AI Act summary 8 and the FSB report on AI and machine learning in financial services 10. For method and its limits: Oreskes and colleagues 5, the scikit-learn guidance on pitfalls 6 and Rudin on interpretable models 9. These are documents, not datasets. Learners should record the edition and date of each text they use.
07Skills that make the knowledge usable
Skills that make the knowledge usable
Knowledge becomes useful through a small set of skills. The discipline treats these as its practical core.
Scoping a model. State its purpose, users and the decisions it informs. Reviewing design. Check assumptions, data and method against the purpose. Testing outcomes. Compare predictions with results, with back-testing where it applies. Benchmarking. Compare with a simpler or independent model. Monitoring. Set thresholds and triggers for review. Reporting. Tell users what the model can and cannot support, in plain terms.
08Open courses and pathways
Open courses and pathways
No open course on this subject was opened at its provider and confirmed as open to the public for this edition, so none is listed.
The supervisory texts are free to read at their issuers, as confirmed on 17 November 2026 1 2 3. The scikit-learn documentation is a practical guide to avoiding leakage when building models 6. A suggested order is the definitions, then one text on validation methods, then a governance text such as SS1/23 or E-23.
None of these is a course with assessment, and none is described here as leading to a qualification.
09How expertise is shown
How expertise is shown
Expertise in this field is shown by work that others can check. The Academy records learning, not credentials.
Finishing a course gives a completion record. It is not a certificate, an accreditation or a qualification, and it does not state that a person is competent in a profession. Achievements on the Academy recognise assessed learning only.
The Academy catalogue lists two courses for this discipline: Model risk and validation: foundations and practice, at the intermediate level, and Model risk and validation: frontier methods, at the advanced level. On 17 November 2026 the catalogue page shows the first as in authoring and the second as planned. Nothing is open for enrolment until it is published in the NXS Learning catalogue, so this guide does not point to an Academy course as a place to enrol.
Outside the Academy, the strongest evidence of expertise is a body of work: a validation report on a small model with scope, tests and findings, a short comparison of two supervisory texts, or a model inventory with owners and review dates. Learners are encouraged to keep such a record from their first course.
10Emerging areas to watch
Emerging areas to watch
Several areas are changing what a model risk professional needs to know. Each is a place to read on once the core is in place.
AI and machine learning. OSFI states that its guideline responds to the surge in AI and machine learning models 3, and the PRA page announces a model risk management roundtable on artificial intelligence and machine learning technologies 2. New effective dates. OSFI Guideline E-23 takes effect on 1 May 2027 3. Revision of frameworks. NIST says AI RMF 1.0 is being revised 7. Legal regimes. The EU AI Act sets risk-based rules for AI systems 8.
Related work across the Nexus network gives wider context. See models and scenarios research at the Global Centre for Risk and Innovation, AI and model risk in finance at the Global Risks Alliance, digital and AI governance at the Global Risks Forum and the proposed Nexus Standards on AI and models. These pages describe programmes and proposals. A link to them implies no endorsement of the sources this guide cites.
What this guide does not claim: it is suggested learning, not a curriculum that anyone must follow, and it is not professional, legal, medical or technical advice. It names programmes, courses and bodies as places to learn and implies no partnership with any of them. It reports figures as their issuers published them and draws no new estimates from them. Risk Academy does not certify competence, and a completion record is not a professional qualification. The Academy records no partners, cohorts or learner outcomes for this faculty, and this guide implies none. The Nexus Standards are proposed and in draft; nothing here treats them as adopted.
Maswali
Maswali na majibu
Where should a complete beginner start in model risk and validation?
Start with stage one and the definitions in the Federal Reserve guidance, then compare them with the OSFI Guideline E-23 definition of model lifecycle.
What is effective challenge?
The Federal Reserve guidance names it as a guiding principle. It is critical review of a model by objective, informed parties who can identify limits and assumptions and cause changes.
Can a model ever be fully validated?
Not in the strict sense. Oreskes and colleagues argue that models of natural systems can be confirmed only partially, so validation builds justified confidence.
Do I need to program to follow this guide?
Not for stages one and three. Stages two and four are easier with a scripting language and some statistics.
Do courses in this discipline lead to a qualification?
No. A completion record shows that you finished a course. It is not a certificate or a professional qualification.
Masharti
Faharasa
- Unda kielelezo
- A quantitative method that processes input data into quantitative estimates.
- Hatari ya kielelezo
- The risk of adverse consequences from decisions based on incorrect or misused model outputs.
- Effective challenge
- Critical review of a model by informed, independent parties who can cause change.
- Back-testing
- Comparing model predictions with what actually happened.
- Benchmarking
- Comparing a model with another model or a simpler method.
- Model lifecycle
- The steps from model design to decommissioning.
Related
Further reading across the network
- Hatari za modeli na uthibitishaji
- Core course of the discipline
- Frontier course of the discipline
- Kitivo cha Sayansi na Mbinu za Hatari
- Learning guide: risk science and methods
- Learning guide: AI and machine learning for risk analysis
- Learning guide: enterprise risk management and risk frameworks
- Learning guide: assurance, internal control and risk culture
- The Global Centre for Risk and Innovation (GCRI): models and scenarios
- The Global Risks Alliance (GRA): AI and model risk
- The Global Risks Forum (GRF): digital and AI governance
- Nexus Standards: AI and models (proposed)
Check your understanding
What are the two main reasons model risk occurs?
Fundamental errors in the model, and incorrect or inappropriate use of the model or its outputs.
What does effective challenge depend on?
A combination of incentives, competence and influence.
Why can a model never be fully validated?
Natural systems are never closed and results are nonunique, so confirmation is partial.
Continue with a course
- Model risk and validation: foundations and practice model-risk-validation.core
- Model risk and validation: frontier methods model-risk-validation.frontier
Vyanzo
References
10 sources, numbered in the order they are first cited
- Board of Governors of the Federal Reserve System (2011-04). Supervisory Guidance on Model Risk Management (SR 11-7, Attachment). www.federalreserve.gov. Accessed 2026-11-17.
- Bank of England, Prudential Regulation Authority (2023-05). SS1/23: Model risk management principles for banks. www.bankofengland.co.uk. Accessed 2026-11-17.
- Office of the Superintendent of Financial Institutions (Canada) (2025-09). Guideline E-23: Model Risk Management (2027). www.osfi-bsif.gc.ca. Accessed 2026-11-17.
- European Central Bank, Banking Supervision (2026). Supervision of internal models. www.bankingsupervision.europa.eu. Accessed 2026-11-17.
- Science (Oreskes, Shrader-Frechette and Belitz) (1994-02). Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences. doi.org. Accessed 2026-11-17.
- scikit-learn developers (2026). Common pitfalls and recommended practices (data leakage). scikit-learn.org. Accessed 2026-11-17.
- U.S. National Institute of Standards and Technology (2026). AI Risk Management Framework. www.nist.gov. Accessed 2026-11-17.
- European Commission (2026). AI Act: regulatory framework for AI. digital-strategy.ec.europa.eu. Accessed 2026-11-17.
- Nature Machine Intelligence (Rudin) (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. doi.org. Accessed 2026-11-17.
- Financial Stability Board (2017-11). Artificial intelligence and machine learning in financial services. www.fsb.org. Accessed 2026-11-17.
Record
Editorial record
Facts and course states are given as of 17 November 2026. This guide is reviewed at least every 270 days and whenever a cited report publishes a new edition or a course of the faculty opens.
Published by the Academy desk of Risk Academy. To report an error or request a correction, write through the Risk Academy contact page; corrections are dated and listed with the page. The Academy's statement on AI use explains how AI assistance is used and reviewed. Text, visuals and data are licensed CC BY 4.0, except where a cited source states other terms.
Cite this article
Academy desk, Risk Academy (2026, November 17). Model risk and validation: a guide to the field. Risk Academy. https://risksacademy.com/risk-science/learning-model-risk-and-validation-guide/
@online{academy2026learningmodelriskand,
author = {{Academy desk, Risk Academy}},
title = {{Model risk and validation: a guide to the field}},
publisher = {Risk Academy},
year = {2026},
month = {11},
url = {https://risksacademy.com/risk-science/learning-model-risk-and-validation-guide/},
language = {en}
} TY - ELEC AU - Academy desk, Risk Academy TI - Model risk and validation: a guide to the field PB - Risk Academy PY - 2026 DA - 2026/11/17 UR - https://risksacademy.com/risk-science/learning-model-risk-and-validation-guide/ LA - en ER -
Republish
Republish or translate this article free of charge, in print or online, under CC BY 4.0: keep the byline and Risk Academy, link to the original and note any change. Figures, maps and twins made by the institutions carry the same licence; third-party data keeps its own credit line.
Attribution to paste
<p>Learning model risk and validation: a guide to the field — Academy desk, Risk Academy, Risk Academy. First published 17 November 2026 at <a href="https://risksacademy.com/risk-science/learning-model-risk-and-validation-guide/">https://risksacademy.com/risk-science/learning-model-risk-and-validation-guide/</a>. Republished under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).</p>