Illustrative: a joint probability density whose dependence shifts over time

Risk Academy, led by the Nexus Consortium · Course catalogue · Risk Science and Methods

Risk Management

Model risk and validation

Model risk and validation, taught twice over: as the working methods of the field and as its frontier since 2024, with labs on the Academy's tools, open data and the Nexus Standards.

Core course Frontier course

The discipline

01Core course · L2 Intermediate

Model risk and validation: foundations and practice

This area covers the risk that decisions rest on models that are wrong or misused, and the controls that manage it: model inventories and governance, validation and back-testing, monitoring and drift, explainability and fairness, third-party and generative AI models, and human-in-the-loop controls.

Level
L2 Intermediate
Study time
about 24 hours (planning estimate)
Status
In authoring
Prerequisites
An L1 course in the area, or an entry review

Why now

It gives model risk functions an international reference for extending validation and monitoring to AI systems. Financial Stability Board, FSB finalises sound practices for the responsible adoption of artificial intelligence (AI), 15 October 2026

Learning outcomes

  1. Define model risk and the other key terms of the discipline and use them correctly in a written case
  2. Explain model risk governance and model validation with a worked example and its source
  3. Apply the discipline's method to a documented case and explain each result with its source
  4. Analyse a named case against model monitoring and drift, stating evidence, date and limits

Modules

  1. Model risk governance
    • Model risk governance
    • Model validation
    • Model monitoring and drift
  2. Explainability and fairness
    • Explainability and fairness
    • Third-party models
    • Generative AI in finance
  3. Human-in-the-loop controls
    • Human-in-the-loop controls
    • Model inventories
  4. Lab: the discipline's method
    • A model that claims a confidence level makes a testable promise about how often outcomes will exceed its figure; counting those exceptions over a test window and comparing the count with the expected distribution gives the verdict, and the verdict feeds the model's tier and its permitted use.
    • Work the method through a documented case and explain each result
    • Read the method's sources and stated values

Labs with Academy tools

Nexus Standards

Teaches

  • NRS-C9 Risk models, methods and validation
  • NRS-I3 Model and data cards for risk
  • NRS-M23 Method fabric: Scientific assurance

Relies on

  • NS-1 The UNI and the artefact catalogue
  • NS-16 UNI application profile (how articles, documents, code, data, APIs and interfaces carry the UNI)
  • NRS-C1 Risk vocabulary and ontology
  • NRS-C2 Risk assessment method
  • NRS-C8 Evidence, uncertainty and provenance

Assessment

  • 40%Lab recordWork the discipline's method through a documented case and explain each result with its source and date
  • 40%Case memoApply the discipline's methods to a named place or organisation, stating evidence and limits
  • 20%Key terms checkShort answers on the key terms, marked against the definitions

Delivery

  • Self-paced online, with the labs in the browser
  • Cohort study over six to eight weeks, with a weekly lab review
  • Co-delivered with a partner institution, inside its own courses

Partner on this course

  • Co-teach a module and bring a current case from your work
  • Host a cohort for your learners, with your own schedule and review
  • Contribute a lab: a dataset your institution publishes openly, or a model
  • Review the course content and its references before it opens
  • Translate the course and adapt its cases to your region

How institutions and experts take part

In the Academy LMS

Course on the learning platform
Intermediate, 4 topics
Cohort group
A community group for each cohort of learners
Achievement
Recorded on completion of the assessed work

References

  1. Financial Stability Board, FSB finalises sound practices for the responsible adoption of artificial intelligence (AI), 15 October 2026
  2. Board of Governors of the Federal Reserve System, SR 11-7: Guidance on Model Risk Management, 2011
  3. Basel Committee on Banking Supervision, Supervisory framework for the use of backtesting in conjunction with the internal models approach, 1996
  4. Bank of England PRA, SS1/23 Model risk management principles for banks, 2023
02Live map

Connected activity in the Nordic and Baltic region

Night-time lights and population density, the footprint of electrified, connected activity, over the Nordic and Baltic region.

The interactive map needs JavaScript; the layer list and sources are below.

Layers 2

Connected activity

Latest available

NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth

Loading the latest NASA imagery.

Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.

Connected activity, framed on the Nordic and Baltic region.Satellite and model products as NASA publishes them, each with its own date. They are indicators, not assessments of any place.

Data: NASA GIBS · ASTER GDEM · Blue Marble · © OpenStreetMap contributors · Natural Earth · NASA EONETSources and licences

  • We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • VIIRS Black Marble night-time lights, NASA Goddard Space Flight Center; distributed by the NASA Level-1 and Atmosphere Archive and Distribution System (LAADS DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • Center for International Earth Science Information Network (CIESIN), Columbia University. 2018. Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11. NASA Socioeconomic Data and Applications Center (SEDAC). Licensed under CC BY 4.0. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). doi:10.7927/H49C6VHW. CC BY 4.0
  • VIIRS near-real-time products (flood, land surface temperature, snow cover, corrected reflectance), NASA LANCE; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • ASTER GDEM is a product of METI and NASA (ASTER Global Digital Elevation Model V003, NASA LP DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). Cite as: NASA/METI/AIST/Japan Spacesystems and U.S./Japan ASTER Science Team (2019). ASTER Global Digital Elevation Model V003. doi:10.5067/ASTER/ASTGTM.003. NASA open data
  • Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com. Public domain
  • © OpenStreetMap contributors (ODbL), via NASA GIBS (coastlines, boundaries, roads and place names). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). ODbL
  • Blue Marble, NASA Earth Observatory; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • Context on the globe and flat map: coastlines, boundaries, roads and place names from NASA GIBS reference layers (OpenStreetMap data, ODbL, © OpenStreetMap contributors); Blue Marble imagery as an optional base; 3D terrain and the elevation read-out from the ASTER GDEM elevation colour index (5 m to 200 m classes, exaggerated for legibility). Boundaries are drawn as published and imply no position on any territory.
  • Globe: MapLibre GL JS 6.9.1, BSD-3-Clause, © MapLibre contributors.
  • Live events: NASA Earth Observatory Natural Event Tracker (EONET); events are indicative and keep their original source links.
  • NASA GIBS: We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS).
  • Optional layers (3 sources in the layer list: live, forecast, observed and context) print their provider credit on the map and here when switched on; forecasts are model output and derived indices are estimates.

Layers and dates

  • Night-time lights, daily: VIIRS Black Marble (VNP46A2), NASA LAADS DAAC · 12 Oct 2026
  • Population density, 2020: Population: CIESIN, Columbia University, Gridded Population of the World v4.11 (2018), NASA SEDAC, CC BY 4.0 · static
  • True colour: VIIRS NOAA-20 corrected reflectance, NASA LANCE · 13 Oct 2026

03How learning runs

How learning runs, as a process you can run

Select a scenario and press Play: course, pathway, practice, faculty, cells and record.

04What is changing

Frontier course · L3 advanced, programme courses and standards

Model risk and validation: frontier methods

The methods, data, tools and studies that changed model risk and validation between 2024 and 2026, taught by reproducing them on open data.

Level
L3 Advanced
Study time
about 28 hours (planning estimate)
Status
Planned
Prerequisites
Model risk and validation: foundations and practice; Working knowledge of a scripting language (Python or R) for the labs

Why now

It gives model risk functions an international reference for extending validation and monitoring to AI systems. Financial Stability Board, FSB finalises sound practices for the responsible adoption of artificial intelligence (AI), 15 October 2026

Learning outcomes

  1. Explain what changed in model risk and validation since 2024 and cite the studies, data or rules behind each change
  2. Reproduce FSB finalised sound practices for adopting AI in finance on open data and state the uncertainty of the result
  3. Evaluate a 2024 to 2026 study: its question, data, method, limits and what it changes for practice
  4. Design an analysis that combines a frontier method with the faculty's instruments, and justify each choice

Modules

  1. FSB finalised sound practices for adopting AI in finance
    • The Financial Stability Board set out 12 sound practices for financial institutions, grouped into organisation-wide AI governance and lifecycle risk management, covering use-case materiality, data governance, explainability, performance evaluation, human oversight and third-party AI dependencies.
    • It gives model risk functions an international reference for extending validation and monitoring to AI systems.
    • Financial Stability Board, 15 October 2026
  2. Internal control guidance is mapped to generative AI risks
    • COSO published Achieving Effective Internal Control Over Generative AI, translating its Internal Control Integrated Framework into guidance for governing generative AI, with risks such as prompt manipulation, opaque reasoning, model drift and frequent configuration changes.
    • It helps validators and auditors frame generative AI controls in a framework boards and auditors already use.
    • COSO, 23 February 2026
  3. US federal AI memo sets minimum practices for high-impact AI
    • OMB memorandum M-25-21 requires agencies to apply minimum risk management practices to high-impact AI, including pre-deployment testing, documented AI impact assessments with independent review, and processes for waivers and monitoring.
    • It defines testing and documentation expectations that suppliers of models to government must be able to evidence.
    • Office of Management and Budget (US), 3 April 2025
  4. UK survey shows heavy reliance on third-party AI models in finance
    • The Bank of England and FCA survey found 75 per cent of firms use AI, a third of use cases are third-party implementations and 46 per cent of firms report only partial understanding of the AI they use.
    • Limited understanding of third-party models is a direct model risk that validation and vendor oversight must address.
    • Bank of England; Financial Conduct Authority, 21 November 2024
  5. Reproduction lab
    • Extend the discipline's method with one frontier method and compare its results
    • State the uncertainty and the limits of the result
    • Peer review against a shared checklist

Labs with Academy tools

Nexus Standards

Teaches

  • NRS-C9 Risk models, methods and validation
  • NRS-I3 Model and data cards for risk
  • NRS-M23 Method fabric: Scientific assurance

Relies on

  • NS-1 The UNI and the artefact catalogue
  • NS-16 UNI application profile (how articles, documents, code, data, APIs and interfaces carry the UNI)
  • NRS-C1 Risk vocabulary and ontology
  • NRS-C2 Risk assessment method
  • NRS-C8 Evidence, uncertainty and provenance

Assessment

  • 50%Method reproductionReproduce one frontier method on open data and state the uncertainty of the result
  • 30%Study reviewA critical review of a 2024 to 2026 study: question, data, method, limits, what it changes
  • 20%Peer reviewReview two other learners' reproductions against a shared checklist

Delivery

  • Self-paced online, with the labs in the browser
  • Cohort study over six to eight weeks, with a weekly lab review
  • Co-delivered with a partner institution, inside its own courses

Partner on this course

  • Co-teach a module and bring a current case from your work
  • Host a cohort for your learners, with your own schedule and review
  • Contribute a lab: a dataset your institution publishes openly, or a model
  • Review the course content and its references before it opens
  • Translate the course and adapt its cases to your region

How institutions and experts take part

In the Academy LMS

Course on the learning platform
Expert, 5 topics
Cohort group
A community group for each cohort of learners
Achievement
Recorded on completion of the assessed work

References

  1. Financial Stability Board, FSB finalises sound practices for the responsible adoption of artificial intelligence (AI), 15 October 2026
  2. COSO, Achieving Effective Internal Control Over Generative AI (GenAI), 23 February 2026
  3. Office of Management and Budget (US), M-25-21: Accelerating Federal Use of AI through Innovation, Governance, and Public Trust, 3 April 2025
  4. Bank of England; Financial Conduct Authority, Artificial intelligence in UK financial services – 2024, 21 November 2024
  5. Board of Governors of the Federal Reserve System, SR 11-7: Guidance on Model Risk Management, 2011
  6. Basel Committee on Banking Supervision, Supervisory framework for the use of backtesting in conjunction with the internal models approach, 1996
  7. Bank of England PRA, SS1/23 Model risk management principles for banks, 2023
05Take part

Take part in the work

Start from who you are.

06Questions

Questions, answers and key terms

Is the course free?
Yes. Every course and learning material of Risk Academy, led by the Nexus Consortium, is free to study, self-paced or with a cohort. Ordinary participation has no fee; paid work, internships and fellowships are separate arrangements.
What does completing a course give me?
A completion record: the course, its version, the date and what was assessed, private by default and shared by you. It stacks into discipline, department and programme achievements. A record states what was studied and assessed and does not certify competence; it is not a licence, a professional title or an employment status.
Where do the data and the places come from?
Every faculty teaches on a twin of a real place built from open data; each layer carries its provider, licence and date, and every result on the Academy states its source. Maps and figures are indicators for teaching, never an assessment of a place.
How do I take part beyond the course?
Form a competence cell of three to seven people, study a shared pathway, take a supervised assignment in the National Working Groups, or bring a cohort from your institution. For organisations.

Key terms

Term

Model risk

The potential for loss or poor decisions from errors in models or their misuse.

Term

Validation

Independent review that a model is sound and fit for its purpose.

Term

Back-testing

Comparing model predictions with what happened.

Term

Drift

Change in data or relationships that degrades a model over time.

ReportsContribute

Contribute evidence from this page.

Reports is the Consortium's quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.

Risk Academy
Related

Around Model risk and validation: courses

Risk Academy

Study model risk and validation with us, or teach it

Every course is free and open. Institutions and experts can co-teach, host a cohort, contribute a lab or review a course before it opens.

Browse the course catalogue Partner with the Academy
Risk Academy

Courses · practice · credentials

Not for profit

Our articles, maps and twins are free to read and free to republish. Readers and partners keep it that way.

Support our work

Who funds us

Risk Academy is a not-for-profit institution. To ask who funds a piece of work, use the funding questions form.

Independence

Editors and the expert reviewers named on each article decide what we publish. Funders and partners do not review or approve articles before publication.

Corrections policy

Licence

Articles: CC BY 4.0. Figures, maps, twins and data: CC BY 4.0. Other uses: ask through the permissions form.

Privacy choices