Illustrative: a joint probability density whose dependence shifts over time

Risk Academy, led by the Nexus Consortium · Risk Science and Methods · Department of Risk Management

Models that are wrong or misused, and the controls that manage them

Model risk and validation

A model is a decision tool with an error term, and model risk is what happens when the error is ignored.

01Scope

What the discipline covers

The subtopics its courses and labs teach.

Courses
In preparation
Levels
L3 Advanced
Labs
In development
  • Model risk governance
  • Model validation
  • Model monitoring and drift
  • Explainability and fairness
  • Third-party models
  • Generative AI in finance
  • Human-in-the-loop controls
  • Model inventories
02Maps

Build a twin of any place

Search any place or open the world; the layers and generators follow model risk and validation on the planet map, the globe and in 3D.

The central Apennines: L'Aquila, Amatrice and Norcia: a flat plan of the faculty twin. Terrain 24 to 2,805 m; 3 reference earthquakes, M6.2 to M6.6; Earthquakes M4.5 and above, past week: read live when the 3D twin opens; USGS ShakeMap intensity contours: read live when the 3D twin opens. Contours every 500 metres from the elevation grid; reference earthquakes as marks.
The central Apennines: L'Aquila, Amatrice and Norcia, Italy. Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices that can draw it, once its data and code have been checked.
  • Terrain 24 to 2,805 m
  • 3 reference earthquakes, M6.2 to M6.6
  • Earthquakes M4.5 and above, past week: read live when the 3D twin opens
  • USGS ShakeMap intensity contours: read live when the 3D twin opens
Credits and dates per layer are listed under the stage. Relief in the twin is exaggerated ×1.5 for legibility.

Drag to move, right drag or Shift with a drag to turn and tilt, Ctrl with the wheel to zoom. Keyboard: arrows move, Shift with arrows turn and tilt, + and – zoom, 0 resets, V steps through the views, Escape clears.

Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices that can draw it, once its data and code have been checked.

The boundaries, names and designations shown do not imply any judgement on the legal status of any territory, or any endorsement or acceptance of such boundaries. Countries: Natural Earth (public domain).

03Live map

Connected activity in the Nordic and Baltic region

Night-time lights and population density, the footprint of electrified, connected activity, over the Nordic and Baltic region.

The interactive map needs JavaScript; the layer list and sources are below.

Layers 2

Connected activity

Latest available

NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth

Loading the latest NASA imagery.

Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.

Connected activity, framed on the Nordic and Baltic region.Satellite and model products as NASA publishes them, each with its own date. They are indicators, not assessments of any place.

Data: NASA GIBS · ASTER GDEM · Blue Marble · © OpenStreetMap contributors · Natural Earth · NASA EONETSources and licences

  • We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • VIIRS Black Marble night-time lights, NASA Goddard Space Flight Center; distributed by the NASA Level-1 and Atmosphere Archive and Distribution System (LAADS DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • Center for International Earth Science Information Network (CIESIN), Columbia University. 2018. Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11. NASA Socioeconomic Data and Applications Center (SEDAC). Licensed under CC BY 4.0. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). doi:10.7927/H49C6VHW. CC BY 4.0
  • VIIRS near-real-time products (flood, land surface temperature, snow cover, corrected reflectance), NASA LANCE; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • ASTER GDEM is a product of METI and NASA (ASTER Global Digital Elevation Model V003, NASA LP DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). Cite as: NASA/METI/AIST/Japan Spacesystems and U.S./Japan ASTER Science Team (2019). ASTER Global Digital Elevation Model V003. doi:10.5067/ASTER/ASTGTM.003. NASA open data
  • Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com. Public domain
  • © OpenStreetMap contributors (ODbL), via NASA GIBS (coastlines, boundaries, roads and place names). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). ODbL
  • Blue Marble, NASA Earth Observatory; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
  • Context on the globe and flat map: coastlines, boundaries, roads and place names from NASA GIBS reference layers (OpenStreetMap data, ODbL, © OpenStreetMap contributors); Blue Marble imagery as an optional base; 3D terrain and the elevation read-out from the ASTER GDEM elevation colour index (5 m to 200 m classes, exaggerated for legibility). Boundaries are drawn as published and imply no position on any territory.
  • Globe: MapLibre GL JS 6.9.1, BSD-3-Clause, © MapLibre contributors.
  • Live events: NASA Earth Observatory Natural Event Tracker (EONET); events are indicative and keep their original source links.
  • NASA GIBS: We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS).
  • Optional layers (3 sources in the layer list: live, forecast, observed and context) print their provider credit on the map and here when switched on; forecasts are model output and derived indices are estimates.

Layers and dates

  • Night-time lights, daily: VIIRS Black Marble (VNP46A2), NASA LAADS DAAC · 12 Oct 2026
  • Population density, 2020: Population: CIESIN, Columbia University, Gridded Population of the World v4.11 (2018), NASA SEDAC, CC BY 4.0 · static
  • True colour: VIIRS NOAA-20 corrected reflectance, NASA LANCE · 13 Oct 2026

04The subject

The subject, methods and measures and debates and bodies

The field and its scope

For model validators, model owners, data scientists and risk committees who must decide whether a model is fit for the decision it supports and keep it that way in use.

05The method

From a model's claim to a back-test verdict: expected exceptions, observed exceptions and the zone the count falls in

A model that claims a confidence level makes a testable promise about how often outcomes will exceed its figure; counting those exceptions over a test window and comparing the count with the expected distribution gives the verdict, and the verdict feeds the model's tier and its permitted use.

Sources: Board of Governors of the Federal Reserve System, SR 11-7: Guidance on Model Risk Management (2011); Basel Committee on Banking Supervision, Supervisory framework for the use of backtesting in conjunction with the internal models approach (1996); Bank of England PRA, SS1/23 Model risk management principles for banks (2023); Box, Science and statistics, Journal of the American Statistical Association (1976); Kupiec, Techniques for verifying the accuracy of risk measurement models, Journal of Derivatives (1995).

06At a glance

At a glance, with 5 related topics

Model risk and validation at a glance

risksacademy.comrisk-sciencediscipline

Model risk and validation: 0 primary courses open, 1 in preparation and 2 also taught here, across 8 subtopics. Courses: RAT-217 L3 in preparation, RAT-207 L2 also taught here, RAT-215 L3 also taught here.L1L2L3L420
Model risk and validation at a glance
  • Open, 0
  • In preparation, 1
  • Also taught here, 2
  • Subtopics, 8
Course codes from the curriculum; study levels L1 to L4.
Twin builderBuild a twin of any placeWorked example: the central Apennines: L'Aquila, Amatrice and Norcia · 4 generators · 7 layers · 4 instruments
07Console

How Model risk and validation is taught, as a process you can run

Press Play to run this process from this page's own data.

08Learning

Learning, practice and connections

Achievements in this discipline

09Take part

Take part in Risk Science and Methods

Start from who you are.

10References

Core references and where to begin

Start with Board of Governors of the Federal Reserve System, then Bank of England Prudential Regulation Authority, then Box.

11Questions

Questions, answers and key terms

Is study here free?
Yes. Every course and learning material of Risk Academy, led by the Nexus Consortium, is free to study, self-paced or with a cohort. Ordinary participation has no fee; paid work, internships and fellowships are separate arrangements.
What does completing a course give me?
A completion record: the course, its version, the date and what was assessed, private by default and shared by you. It stacks into discipline, department and programme achievements. A record states what was studied and assessed and does not certify competence; it is not a licence, a professional title or an employment status.
Where do the data and the places come from?
Every faculty teaches on a twin of a real place built from open data; each layer carries its provider, licence and date, and every result on the Academy states its source. Maps and figures are indicators for teaching, never an assessment of a place.
How do I take part beyond the courses in model risk and validation?
Form a competence cell of three to seven people, study a shared pathway, take a supervised assignment in the National Working Groups, or bring a cohort from your institution. For organisations.

Key terms

Term

Model risk

The potential for loss or poor decisions from errors in models or their misuse.

Term

Validation

Independent review that a model is sound and fit for its purpose.

Term

Back-testing

Comparing model predictions with what happened.

Term

Drift

Change in data or relationships that degrades a model over time.

ReportsContribute

Contribute evidence from this page.

Reports is the Consortium's quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.

Risk Academy
Model risk and validation

Prepare with a related course

Open courses in connected disciplines teach the methods this discipline will build on.

Risk Academy

Courses · practice · credentials

Not for profit

Our articles, maps and twins are free to read and free to republish. Readers and partners keep it that way.

Partner with us

Who funds us

Risk Academy is a not-for-profit institution. To ask who funds a piece of work, use the funding questions form.

Independence

Editors and the expert reviewers named on each article decide what we publish. Funders and partners do not review or approve articles before publication.

Licence

Articles: CC BY 4.0. Figures, maps, twins and data: CC BY 4.0. Other uses: ask through the permissions form.