Risk Academy, led by the Nexus Consortium · Risk Science and Methods · Department of Risk Management
Models that are wrong or misused, and the controls that manage them
Ubungozi bamamodeli nokuqinisekisa
A model is a decision tool with an error term, and model risk is what happens when the error is ignored.
Map model risk and validation on a real placeSee the courses
Illustrative: a joint probability density whose dependence shifts over time
01Ububanzi
What the discipline covers
The subtopics its courses and labs teach.
- Department
- Department of Risk Management
- Izifundo
- In preparation
- Amazinga
- L3 Advanced
- Amalabhorethri
- In development
- Model risk governance
- Model validation
- Model monitoring and drift
- Explainability and fairness
- Third-party models
- Generative AI in finance
- Human-in-the-loop controls
- Model inventories
02Amamephu
Build a twin of any place
Search any place or open the world; the layers and generators follow model risk and validation on the planet map, the globe and in 3D.
- Terrain 24 to 2,805 m
- 3 reference earthquakes, M6.2 to M6.6
- Earthquakes M4.5 and above, past week: read live when the 3D twin opens
- USGS ShakeMap intensity contours: read live when the 3D twin opens
Drag to move, right drag or Shift with a drag to turn and tilt, Ctrl with the wheel to zoom. Keyboard: arrows move, Shift with arrows turn and tilt, + and – zoom, 0 resets, V steps through the views, Escape clears.
Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices that can draw it, once its data and code have been checked.
The boundaries, names and designations shown do not imply any judgement on the legal status of any territory, or any endorsement or acceptance of such boundaries. Countries: Natural Earth (public domain).
03The subject
The subject, methods and measures and debates and bodies
The discipline
The field and its scope
For model validators, model owners, data scientists and risk committees who must decide whether a model is fit for the decision it supports and keep it that way in use.
Model risk and validation study the losses and harms that follow when decisions rest on models that are wrong, misused or no longer fit the world, and the controls that reduce that risk: inventories, tiering by materiality, independent validation, performance monitoring, change control and clear limits on use. The field took shape in banking after pricing and credit model failures and was codified in the Federal Reserve's SR 11-7 (2011) and the Bank of England's SS1/23 (2023); its methods now apply to insurance, catastrophe, climate, epidemiological and machine learning models, including generative systems whose outputs are hard to test against a ground truth.
The discipline is organised around the model life cycle. Development documents purpose, data, assumptions and limitations; validation tests conceptual soundness, implementation and outcomes with back-testing, benchmarking and sensitivity analysis; monitoring tracks drift in inputs and performance; governance records the inventory, the tiering and who may approve what. Explainability and fairness testing enter where models affect people. The standing questions are what validation can show for models of rare events, how much independence is enough, and how to govern third-party and foundation models whose internals are not available to the user.
Measures
Quantities and units
| Quantity | Unit or scale | What to know |
|---|---|---|
| Back-test exceedance rate | Count of exceptions per period against the expected count at the model's confidence level | For a quantile forecast such as value at risk, the number of times loss exceeded the forecast. Tests such as Kupiec's are weak for small samples: a year of daily data cannot reject a 99 per cent model whose true rate is 1.5 per cent. |
| Discrimination (AUC, Gini) and calibration (expected versus observed rate) | Dimensionless, 0.5 to 1 for AUC; ratio or calibration slope for calibration | Standard performance statistics for classification models. Both are estimated from a sample with a confidence interval and both degrade when the population shifts; discrimination can stay high while calibration fails. |
| Population stability index and feature drift statistics | Dimensionless index; or a test statistic per feature | Measures of how far the input distribution has moved from the development sample. Thresholds in use are conventions, not derived from loss consequences. |
| Model materiality tier | Ordinal tier from exposure, complexity and use | Drives the depth and frequency of validation. It is a governance judgement; the uncertainty lies in whether exposure through a model is known at all, especially for models embedded in other models. |
| Sensitivity of output to assumptions | Change in output per unit change in a parameter, or a Sobol index share of variance | Shows which assumptions matter. Local sensitivities miss interactions; global sensitivity analysis needs many model runs that slow models cannot afford. |
| Validation finding count by severity and age | Counts; days open | The operational state of the control. A finding backlog with long ages signals that validation has no teeth; the count alone says nothing about the risk the findings carry. |
Izindlela
Methods and their limits
Each method is listed with what it assumes and where it fails.
| Indlela | Assumes | Fails when |
|---|---|---|
| Independent validation (conceptual soundness, implementation testing, outcomes analysis) | That a team outside the model's development line can read the documentation, reproduce key results, test the code and judge the assumptions against the use. | When validators lack the domain knowledge or the data to reproduce, when the model is a black box from a vendor, or when validation becomes a document review that never runs the model. |
| Back-testing and out-of-sample testing | That past outcomes are a fair test of the model's future use, that enough independent observations exist, and that the test is set before the results are seen. | For rare events where the test has no power, when the data were used in development, or when the environment changed so that past performance no longer bears on the future. |
| Benchmarking and challenger models | That a simpler or alternative model, run on the same data, exposes where the production model's assumptions drive its output. | When the benchmark shares the production model's data and blind spots, or when agreement between models is read as validation against reality. |
| Monitoring for drift and performance decay | That input distributions and outcomes are observed in use with little delay, and that thresholds trigger review or retraining. | When outcomes arrive years later (credit default, climate), when the outcome is itself shaped by the model's decisions, or when thresholds are set without reference to the harm from a stale model. |
| Explainability and fairness testing | That feature attributions (for example Shapley values) and group performance comparisons reveal whether a model relies on acceptable factors and treats groups comparably. | When attributions are unstable across methods, when fairness metrics conflict and no rule chooses between them, or when the protected attribute is not recorded and proxies cannot be traced. |
| Model inventory with tiering and use restrictions | That every model can be found, described, assigned an owner and a tier, and that use beyond the documented purpose is detectable. | When spreadsheets, embedded rules and third-party components are not counted as models, or when the inventory is a list without links to validation evidence. |
Debates
What the field debates
- Should high-stakes models be interpretable by design, or is post-hoc explanation of complex models enough?
- Rudin (2019) argues that interpretable models perform comparably on most tabular problems and that post-hoc explanations are approximations of an approximation. The other side points to tasks (images, text, weather) where complex models are far more accurate, to supervisory acceptance of explanation methods, and holds that the requirement is reliable testing and monitoring rather than a particular model class.
- What does validation mean for models of rare events that cannot be back-tested?
- One view, following Oreskes and co-authors (1994), holds that such models can be confirmed in part but never validated, so the control is transparency of assumptions and sensitivity. The other holds that component-level testing, benchmarking and historical event reconstruction give adequate evidence, and that refusing the word validation leaves decision makers without a usable verdict.
- Can generative and foundation models be governed under existing model risk frameworks?
- Supervisors and SS1/23 hold that the principles apply: define the use, test for it, monitor and restrict. Critics reply that the user cannot inspect training data or weights, that outputs are open-ended so outcome testing is partial, and that NIST's AI Risk Management Framework and ISO/IEC 42001 add the missing elements of use-case governance and incident reporting; the evidence base of documented failures is still young.
Bodies and data
Bodies that set practice, and the data they publish
Bodies and standards are named so learners can find and apply their published work; the Academy teaches their use and links to the publisher, and no affiliation is implied.
| Body | Role in the field |
|---|---|
| Board of Governors of the Federal Reserve System and the Office of the Comptroller of the Currency | Issued SR 11-7 / OCC 2011-12, the supervisory guidance on model risk management that defined the field for banks. |
| Bank of England Prudential Regulation Authority | Issued supervisory statement SS1/23 on model risk management principles for banks, in force from May 2024, extending the scope to all models including those using artificial intelligence. |
| National Institute of Standards and Technology (NIST) | Publishes the AI Risk Management Framework (AI 100-1) and its generative AI profile, used as a reference for governing machine learning models beyond finance. |
| ISO/IEC JTC 1/SC 42 Artificial intelligence | Maintains ISO/IEC 42001 (AI management systems), ISO/IEC 23894 (AI risk management guidance) and related standards on data quality and bias. |
| European Banking Authority and European Central Bank | Set model requirements for internal ratings and run the targeted review of internal models that tests how banks' models are validated. |
| Dataset | Steward |
|---|---|
| AI Incident Database | Responsible AI Collaborative |
| OECD AI Incidents Monitor | Organisation for Economic Co-operation and Development |
| Supervisory stress test scenarios and results | Board of Governors of the Federal Reserve System |
04Indlela
From a model's claim to a back-test verdict: expected exceptions, observed exceptions and the zone the count falls in
A model that claims a confidence level makes a testable promise about how often outcomes will exceed its figure; counting those exceptions over a test window and comparing the count with the expected distribution gives the verdict, and the verdict feeds the model's tier and its permitted use.
Sources: Board of Governors of the Federal Reserve System, SR 11-7: Guidance on Model Risk Management (2011); Basel Committee on Banking Supervision, Supervisory framework for the use of backtesting in conjunction with the internal models approach (1996); Bank of England PRA, SS1/23 Model risk management principles for banks (2023); Box, Science and statistics, Journal of the American Statistical Association (1976); Kupiec, Techniques for verifying the accuracy of risk measurement models, Journal of Derivatives (1995).
05At a glance
At a glance, with 5 related topics
- Build a twin of any placeTwin builderWorked example: the central Apennines: L'Aquila, Amatrice and Norcia · 4 generators · 7 layers · 4 instruments
Igrafu ye-3D
Ifakhalthi Yesayensi Yobungozi Nezindlela
265 nodes1,691 relations9 views
06Ikhonsoli
How Model risk and validation is taught, as a process you can run
Press Play to run this process from this page's own data.
Ubungozi bamamodeli nokuqinisekisa
How Model risk and validation is taught
5 subjects in scope, 3 key terms, courses in development, 1 lab, in the Risk Management department of Risk Science and Methods: from the scope to the method and its terms, the courses that teach it, the labs on the twin, the assessment and the record.
-
01 Scope
- 01
Model risk governance
Start · the discipline
Model risk governance
- 02
Model validation
Start · the discipline
Model validation
- 03
Model monitoring and…
Start · the discipline
Model monitoring and drift
- 04
Explainability and…
Start · the discipline
Explainability and fairness
- 05
Third-party models
Start · the discipline
Third-party models
- 01
-
02 Method and terms
- 06
From a model's claim to…
Model · the learner
A model that claims a confidence level makes a testable promise about how often outcomes will exceed its figure; counting those exceptions…
- 07
Ingozi yemodeli
Record · the discipline
The potential for loss or poor decisions from errors in models or their misuse.
- 09
Validation
Record · the discipline
Independent review that a model is sound and fit for its purpose.
- 11
Back-testing
Record · the discipline
Comparing model predictions with what happened.
- 06
-
03 Courses
- 08
RAT-217 Model Risk…
Record · the learner
Model Risk Management and Validation
- 08
-
04 Labs
- 10
Umfanekiso wedijithali
Exercise · the learner, on the faculty twin
The method runs on the faculty twin and the admitted map layers.
- 10
-
05 Assess
- 12
Module checks
Gate · the course
Each module ends with a short check against its outcomes.
- 13
Assessed work
Decision · the learner
The assessed work that closes a course: a lab record, a case memo or an assignment, reviewed against the rubric.
- 12
-
06 Record
- 14
Irekhodi lokuphothula
Outcome · the learner
The record stacks into the discipline achievement.
- 15
Discipline achievement
Record · the Academy
The discipline achievement follows the discipline rule, read from the records.
- 14
Model risk and validation · Risk Management An outline of how the work is designed to run; it does not report the status of any decision or record.
07Ukufunda
Learning, practice and connections
Impumelelo
Achievements in this discipline
Course achievements
One for each course with this as its primary discipline, earned when you pass the course's assessed activities.
Discipline: Model risk and validation
Complete 3 assessed courses in model risk and validation, at least 2 with it as their primary discipline and at least 1 at Intermediate (L2) or above, totalling at least 45 measured learning hours. In development.
Department: Risk Management
Earn two discipline achievements in the Department of Risk Management. In development.
Programme: Nexus-Based Risk Management flagship pathway
Complete every core course of the Nexus-Based Risk Management flagship pathway under assessed completion, ending with its capstone. In development.
An achievement is a record of the learning completed and assessed here. How recognition works.
Izindlela
Pathways that teach it
- Nexus-Based Risk Management flagship pathwayAcademy flagship20 courses across the faculties.
- Enterprise risk managerRole pathwayOne of its 12 courses is in this discipline.
- Infrastructure resilience engineerRole pathwayOne of its 10 courses is in this discipline.
- Risk analystRole pathwayOne of its 11 courses is in this discipline.
Where this is practised
Where the subject is practised
- The Global Risks Alliance (GRA)AI and Financial Model RiskCredit and pricing models, Explainability and fairness, Generative AI in finance, Human in the loop controls, Model monitoring and drift, Model risk governance, Stress and scenario models, Third party models
Field platforms of the ecosystem where the subject is studied and applied.
Practised in
The three Foundry systems where it is practised
- FoundryNXS-NXQ systemFitness of models for a task
- NXS-LAB systemModel comparison
- NXS-GRX systemValidation studies behind risk products
Foundry system pages describe what each system and its rooms are designed to do; the courses teach the methods these systems apply.
Across the ecosystem
Across the ecosystem
The science, platforms and standards this discipline draws on and contributes to across the ecosystem.
Foundations in the STEM Commons
- The Global Centre for Risk and Innovation (GCRI Canada)Uncertainty quantification and model validationUncertainty quantification and model validation characterise how uncertainty in inputs, parameters and model structure affects predictions.
- Probability and statisticsProbability and statistics provide the mathematical framework for describing uncertainty and learning from data.
- AI evaluation and assurance engineeringAI evaluation and assurance engineering develops methods to test, monitor and provide justified confidence in AI systems.
Standards for the field
- Nexus StandardsDigital, Cyber and AIThe subsite that keeps the standards this discipline teaches.
- Data and EvidenceThe subsite that keeps the standards this discipline teaches.
Pages of the ecosystem, cited so learners can follow the field into research, platforms, standards and national practice.
08Bamba iqhaza
Take part in Risk Science and Methods
Qala kulokho ongikho.
Izindlela
Indlela
Izikhungo zikazwelonke
Iminyango kahulumeni, amahhovisi ezibalo nama-ejensi: idesiki likazwelonke nesahluko, nesihlalo emaqenjini okusebenza. Izahluko zikazwelonke
Indlela
Imikhakha nemisebenzi
Operators, regulators, banks, insurers and professional bodies whose work touches risk science and methods. For organisations
Indlela
Ukufunda
Izifundo nokuzijwayeza, nerekhodi lazo zonke izifundo eziqediwe. Mahhala ukufunda. Ukufunda okuvulekile
Indlela
Amazinga
Bhala okusalungiswa, buyekeza futhi wamukele izindlela namazinga edatha namaqembu okusebenza. Amaqembu okusebenza
Indlela
Imiphakathi nocwaningo
Universities, civil society and individual experts: projects, data and review. Write to the Academy
Izimemo ezivulekile
| Isimemo esivulekile | Kufunwa | Kuvalwa |
|---|---|---|
| Isimemo sokusungula sochwepheshe emaqenjini okusebenza CALL-FOUNDING-EXPERTS | uchwepheshe, umhleli, usihlalo | 2026-12-15 |
| Ukuqokwa okuvulekile kwezihlalo ezintathu zochwepheshe abazimele ze-Global Standards Council CALL-GSC-IND | ilungu lomkhandlu | 2026-12-01 |
| Ukuqokwa kwamalungu esifunda esikhashana (kusebenza kuphela uma isinqumo sesikhashana sithathwa) CALL-GSC-REG | ilungu lomkhandlu | 2026-12-15 |
| Ukubuyekezwa kwezinhlaka zomongo wengozi wegagasi 1 (C1, C2, C3, C4, C8, C10, C11) CALL-REVIEW-WAVE1 | umnikeli | 2027-01-31 |
| Ochwepheshe bamazinga ayisisekelo esayensi yengozi: ukungaqiniseki nokulinganiswa kobufakazi, izilinganiso, ukwahlulela kochwepheshe okuhlelekile, ukubekezeleleka, izimo ezeqisayo nezehlakalo ezihlanganisiwe, ingozi eyinhlekelele neyokuphila, izimiso nokulingana, amayunithi nokulinganisa (NRS-C17 kuya ku-NRS-C24) CALL-CORE-SCIENCE | uchwepheshe, umhleli, usihlalo | 2027-01-31 |
| Ababuyekezi abazimele bemindeni ye-atlas yezingozi, abathathu abavela okungenani ezifundeni ezimbili emndenini ngamunye (NHS2-R19), imindeni ehamba phambili kuqala CALL-ATLAS-REVIEW | uchwepheshe | 2027-03-31 |
| Amalungu e-Frontier Safety Review Panel yesikhashana nababuyekezi beqembu elibomvu (ukuphepha kwezinto eziphilayo nokuvikeleka kwazo, ukuphepha nokuvikeleka kwe-AI, ukuvikeleka kwengqalasizinda ebalulekile); izicelo ziphathwa ngokuyimfihlo CALL-FRONTIER-SAFETY-REVIEW | uchwepheshe | 2026-11-30 |
| Ochwepheshe be-ontology, i-metrology, ukuqinisekiswa kwesofthiwe nokuhlolwa kokuhambisana kwerejista yezidingo, ukumelwa komshini kwe-NHS-2, amasethi okuhlola ukuhambisana nohlelo lokuhlola ukuhambisana CALL-CONFORMANCE-ONTOLOGY | uchwepheshe, umhleli | 2027-01-31 |
Izimemo ezidlule usuku lwazo lokuvala zifihlwa ngokuzenzakalelayo. Zonke izimemo, iziphakamiso nezindlela: izindawo nezimemo ezivulekile.
Amaseli nama-CERT
Qoqa iseli
Hlanganisa abantu abathathu kuya kwabayisikhombisa abanamakhono ahambisanayo: idatha, ukusebenza enkundleni, inqubomgomo nokuxhumana.
Libhalise
Write to the team with the cell's name, members, area and region. You receive a reference and a contact. Write to the Academy
Khetha umqulu
Khetha ukugxila okukodwa: ucwaningo nobufakazi, idatha namamephu, ukubuyekezwa kwamazinga, noma ukulungiselela komphakathi.
Funda ndawonye
Landela izindlela zokufunda ezabelwana ngazo e-Risk Academy, nokuzijwayeza nerekhodi lazo zonke izifundo eziqediwe. Izinsongo nokuqina
Hola futhi ubike
Hold regular sessions. Publish what the cell learns through Reports. Bring findings to the working groups. Working groups
Gxila njengeThimba Lomphakathi Lokusabela Ezimweni Eziphuthumayo (CERT)
A CERT prepares its neighbourhood for the hazards it faces. It builds a local hazard profile from this site's map. It trains in household and street preparedness. It learns first aid and light search through recognised local providers, and it runs drills. A CERT works alongside the local emergency services and follows their direction. It never replaces them and is not an emergency service.
I-Index
Lokho ithimba likazwelonke noma umphakathi wendawo ongakwenza ku-Global Risks Index (NXS-GRX) namuhla.
NXS-GRX
Your country's profile
Amaphrofayela engozi avulekile amazwe angu-195, nemithombo namalayisense kuyo yonke inani. Vula Inkomba
NXS-GRX
Lungisa noma uphendule
Lungisa inani noma uphendule ngezwe lakho. Ushintsho lurekhodwa kanye nomphumela walo. Vula Inkomba
NXS-GRX
Nikela ngedatha
Nikela ngeqoqo ledatha. Lingena ngesango lelayisense, ubongwa. Vula Inkomba
NXS-GRX
Sebenzisa idatha evulekile
Izendlalelo, amaphrofayela namathayela nge-API evulekile. Vula Inkomba
09References
Core references and where to begin
Start with Board of Governors of the Federal Reserve System, then Bank of England Prudential Regulation Authority, then Box.
- Board of Governors of the Federal Reserve System, SR 11-7: Guidance on Model Risk Management (2011). federalreserve.gov
- Bank of England Prudential Regulation Authority, SS1/23 Model risk management principles for banks (2023). bankofengland.co.uk
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023). DOI
- Box, G. E. P., Science and statistics, Journal of the American Statistical Association (1976). DOI
- Oreskes, N., Shrader-Frechette, K. and Belitz, K., Verification, validation, and confirmation of numerical models in the earth sciences, Science (1994). DOI
- Rudin, C., Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nature Machine Intelligence (2019). DOI
- Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M. and Bouchachia, A., A survey on concept drift adaptation, ACM Computing Surveys (2014). DOI
- Mitchell, M. et al., Model cards for model reporting, Proceedings of the Conference on Fairness, Accountability, and Transparency (2019). DOI
- International Organization for Standardization and IEC, ISO/IEC 42001:2023 Artificial intelligence management system (2023). iso.org
10Imibuzo
Imibuzo, izimpendulo namagama abalulekile
Ukufunda lapha kungamahhala yini?
Yebo. Zonke izinkambo nezinto zokufunda ze-Risk Academy, eholwa yi-Nexus Consortium, zifundwa mahhala, ngesivinini sakho noma neqembu. Ukubamba iqhaza okujwayelekile akunayo inkokhelo; umsebenzi ohlawulelwayo, ukuqeqeshwa emsebenzini nezibonelelo zocwaningo kuyizinhlelo ezihlukile.
Kuyini engikuthola ngokuphothula isifundo?
Irekhodi lokuphothula: isifundo, inguqulo yaso, usuku nokuhlolwe, eliyimfihlo ngokuzenzakalelayo futhi labelwana nguwe. Lihlangana libe yimiphumela yomkhakha, yomnyango neyohlelo. Irekhodi likhomba okufundiwe nokuhlolwe, alisho ukuthi ukhono luqinisekisiwe; akulona ilayisense, isihloko sobuchwepheshe noma isimo somsebenzi.
Idatha nezindawo zivelaphi?
Ifakhalthi ngayinye ifundisa ngamawele endawo yangempela akhiwe ngedatha evulekile; isendlalelo ngasinye siveza umhlinzeki waso, ilayisense nosuku, futhi umphumela ngamunye ku-Academy uveza umthombo wawo. Amamephu nezibalo yizinkombandlela zokufundisa, ayisona isilinganiso sendawo.
How do I take part beyond the courses in model risk and validation?
Yakha iseli lekhono labantu abathathu kuya kwabayisikhombisa, funda indlela okwabelwana ngayo, thatha umsebenzi ogadiwe kuma-National Working Groups, noma ulethe iqembu labafundi elivela esikhungweni sakho. Ezinhlanganweni.
Amagama abalulekile
- Ingozi yemodeli
- The potential for loss or poor decisions from errors in models or their misuse.
- Validation
- Independent review that a model is sound and fit for its purpose.
- Back-testing
- Comparing model predictions with what happened.
- Drift
- Change in data or relationships that degrades a model over time.
ImibikoNikela
Nikela ngobufakazi kusuka kuleli khasi.
I-Reports iyincwadi yekota ye-Consortium. Lokho oakwaziyo ngalesi sihloko kungaba ubufakazi, icala noma ukulungisa kuhlelo olulandelayo, kubongwe wena futhi kushicilelwe ngaphansi kwe-CC BY 4.0 lapho kungumsebenzi wakho.
Risk Academy
- EmailGateway GW-ACADEMY Send it from this page Evidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen deposit Deposit it and submit to the community Publish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsRisk Academy See every way to contribute The contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Ungalokothi uthumele idatha yomuntu siqu, imininingwane ebucayi yengqalasizinda noma noma yini ongenalo ilungelo lokuyabelana ngayo. Umnikelo ubuyekezwa ngaphambi kokusetshenziswa; ukuwuthumela akunikezi ubuqambi noma amalungelo okuhlela, futhi ukusekela akuthengi muphumela.
Ubungozi bamamodeli nokuqinisekisa
Prepare with a related course
Open courses in connected disciplines teach the methods this discipline will build on.
Buka ikhathalogi yezifundoBack to Department of Risk Management