Risk Academy, led by the Nexus Consortium · Risk Science and Methods · Department of Quantitative Risk Analysis
Learning from data without hiding the error
I-AI nokufunda koomatshini kuhlalutyo lomngcipheko
Machine learning entered risk analysis through its methods, not its headlines: emulators that stand in for slow hazard and catastrophe models, classifiers of damage from imagery, anomaly detectors on sensor streams and foundation models trained on satellite archives.
Map aI and machine learning for risk analysis on a real placeSee the courses
Illustrative: layers of units and their weighted connections, an activation front sweeping from input to output
01Ubungakanani bomsebenzi
What the discipline covers
The subtopics its courses and labs teach.
- Department
- Department of Quantitative Risk Analysis
- Izifundo
- In preparation
- Amanqanaba
- In development
- Iilabhoratri
- In development
- Supervised learning for risk outcomes
- Emulators and surrogates of physical and catastrophe models
- Uncertainty quantification in machine learning
- Conformal prediction and distribution-free coverage
- Anomaly detection and change detection
- Geospatial foundation models and satellite embeddings
- Causal inference and machine learning
- Validation, drift monitoring and model risk for machine learning
- Fairness and bias in risk models
02Iimephu
Build a twin of any place
Search any place or open the world; the layers and generators follow aI and machine learning for risk analysis on the planet map, the globe and in 3D.
- Terrain 24 to 2,805 m
- 3 reference earthquakes, M6.2 to M6.6
- Earthquakes M4.5 and above, past week: read live when the 3D twin opens
- USGS ShakeMap intensity contours: read live when the 3D twin opens
Drag to move, right drag or Shift with a drag to turn and tilt, Ctrl with the wheel to zoom. Keyboard: arrows move, Shift with arrows turn and tilt, + and – zoom, 0 resets, V steps through the views, Escape clears.
Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices that can draw it, once its data and code have been checked.
The boundaries, names and designations shown do not imply any judgement on the legal status of any territory, or any endorsement or acceptance of such boundaries. Countries: Natural Earth (public domain).
03The subject
The subject, methods and measures and debates and bodies
The discipline
The field and its scope
For risk analysts and data scientists who use or review machine learning in a risk estimate and need to know when its outputs can be trusted and how its errors propagate.
Machine learning for risk analysis studies the use of learned models to classify and predict risk outcomes, to emulate slow physical and catastrophe models, to detect anomalies and change, and to quantify the uncertainty of those outputs so that they can enter a risk estimate honestly. The field sits between statistics, computer science and the domain sciences: a flood emulator, a default classifier and a satellite damage detector share the same questions about training data, generalisation, calibration and drift. Its users are modellers who need speed, analysts who need predictions where no physical model exists, and validators who must decide whether a learned model is fit for the decision.
The discipline is organised around the learning task (supervised, unsupervised, causal), the uncertainty tools (Bayesian and ensemble methods, conformal prediction with distribution-free coverage, calibration), the validation regime (held-out tests, temporal and spatial splits, drift monitoring) and the governance of fairness and explainability where models affect people. Geospatial foundation models and satellite embeddings have brought pre-trained representations into exposure and damage mapping. The field's central tension is between accuracy on the training distribution and reliability where risk lives: in the tail, after a regime change, and in places with no labels.
Measures
Quantities and units
| Quantity | Unit or scale | What to know |
|---|---|---|
| Out-of-sample error (RMSE, log loss, AUC) | Variable's unit for RMSE; dimensionless for log loss and AUC | Performance on data not used in training. The value depends on how the split was made: random splits overstate performance when neighbouring points are correlated in space or time, so spatial and temporal blocking is the fair test for risk applications. |
| Calibration error | Expected calibration error, dimensionless; or reliability diagram deviation | Whether a predicted probability of 0.2 verifies about one time in five. Guo and co-authors (2017) show modern neural networks are often over-confident; temperature scaling and similar methods correct it on the validation distribution only. |
| Prediction interval coverage and width | Proportion of outcomes inside the interval at a nominal level; interval width in the variable's unit | Conformal prediction guarantees marginal coverage under exchangeability without a model assumption; coverage can still fail for subgroups or after a shift, and narrow intervals with correct coverage are the aim. |
| Emulator error relative to the full model | Same unit as the model output, often as a percentage of the output range | How closely a surrogate reproduces a physical or catastrophe model. The error is measured on held-out model runs, not on reality; the emulator inherits the full model's errors and adds its own, which are largest at the extremes of the parameter space. |
| Drift statistics | Population stability index, Kolmogorov-Smirnov statistic or embedding distance, dimensionless | Shift in inputs or predictions since training. Input drift without outcome labels cannot say whether performance has fallen; it says only that the model is being asked questions it was not trained on. |
| Group performance disparity | Difference or ratio of error rates, false positive rates or calibration between groups, dimensionless | Fairness measures. Several definitions cannot all hold at once when base rates differ, so the choice of measure is a policy decision that should be stated. |
Iindlela zokusebenza
Methods and their limits
Each method is listed with what it assumes and where it fails.
| Indlela yokusebenza | Assumes | Fails when |
|---|---|---|
| Supervised learning (gradient boosting, random forests, neural networks) | That labelled examples represent the deployment population, that the relation between features and outcome is stable, and that the loss function matches the decision cost. | When labels are scarce or biased (loss data recorded only where insurance exists), when the model learns a proxy that breaks under intervention, or when performance in the tail, where risk lives, is averaged away by the bulk. |
| Emulation and surrogate modelling (Gaussian processes, neural surrogates) | That the full model's response is smooth enough to learn from a designed set of runs, and that the emulator's own uncertainty can be estimated and carried forward (Kennedy and O'Hagan, 2001). | At the edges of the design space, for discontinuous responses (levee failure, phase changes), or when the emulator is used to explore extremes the full model was never run at. |
| Uncertainty quantification (deep ensembles, Bayesian approximations, dropout) | That disagreement among models or posterior samples reflects epistemic uncertainty and that the aleatoric part can be learned from the data. | When all ensemble members share the training data's blind spots, when approximate posteriors are over-confident, or when the uncertainty estimate is itself never validated against held-out coverage. |
| Conformal prediction | That calibration and test data are exchangeable, and that a reasonable score function is available from any base model. | Under distribution shift that breaks exchangeability, for coverage conditional on a subgroup (only marginal coverage is assured), and when the calibration set is small so that intervals are wide. |
| Anomaly and change detection | That normal behaviour can be characterised from history and that departures are rare, detectable and worth investigating. | When the training period includes the anomalies, when the base rate of true events is so low that the false alarm count swamps analysts, or when the detector flags change in the sensor rather than in the world. |
| Causal inference with machine learning (double machine learning, causal forests) | That the causal structure can be stated (a directed graph), that confounders are observed, and that treatment and outcome are measured without the model's own feedback. | When unobserved confounding is plausible, when the positivity assumption fails for the groups of interest, or when the question is predictive and causal machinery adds assumptions without benefit. |
Debates
What the field debates
- Should machine learning emulators replace physical models in risk assessment?
- Proponents point to speed gains of orders of magnitude that make probabilistic assessment with many scenarios feasible, and to benchmark results in weather forecasting. Critics reply that emulators cannot be trusted beyond the training envelope, that extremes are exactly where the training data are sparse, and that an emulator's error relative to the full model is not an error relative to reality; hybrid designs that keep physics in the loop are the current compromise.
- Is distribution-free uncertainty (conformal prediction) enough, or are Bayesian models needed?
- The conformal side holds that finite-sample coverage guarantees with no model assumptions are what a risk analyst needs and that Bayesian posteriors depend on priors and approximations that are hard to check. The Bayesian side replies that marginal coverage is weak, that decisions need full predictive distributions and conditional uncertainty, and that conformal intervals say nothing about why the model is uncertain.
- Which fairness definition should a risk model satisfy?
- Calibration within groups is defended as the condition for a score to mean the same thing for everyone. Equalised error rates are defended as the condition for the model to burden groups equally. Impossibility results show both cannot hold when base rates differ, so one position argues for context-specific choice disclosed in documentation, while another argues that certain uses should not be automated at all.
Bodies and data
Bodies that set practice, and the data they publish
Bodies and standards are named so learners can find and apply their published work; the Academy teaches their use and links to the publisher, and no affiliation is implied.
| Body | Role in the field |
|---|---|
| National Institute of Standards and Technology (NIST) | Publishes the AI Risk Management Framework and runs evaluations and measurement programmes for machine learning systems. |
| ISO/IEC JTC 1/SC 42 Artificial intelligence | Maintains the international standards on AI management systems, risk management, bias and data quality for machine learning. |
| International Machine Learning Society and the NeurIPS and ICML communities | Peer-reviewed venues where the methods for uncertainty quantification, calibration and conformal prediction are published and benchmarked. |
| Royal Statistical Society | Professional body for the statistical foundations of learning, calibration and evaluation, with a data science and AI section. |
| IEEE Geoscience and Remote Sensing Society | The learned society for machine learning on Earth observation data, including the benchmarks for damage, flood and land cover mapping. |
| Dataset | Steward |
|---|---|
| OpenML benchmark datasets and tasks | OpenML Foundation |
| Prithvi geospatial foundation models | IBM and NASA on Hugging Face |
| AI Incident Database | Responsible AI Collaborative |
| SpaceNet open satellite imagery datasets | SpaceNet |
04Indlela
From a trained model to an interval with assured coverage: residuals, the conformal quantile and the coverage a user can count on
A learned model gives a point prediction with no promise attached; holding out a calibration set, ranking its residuals and taking a quantile wraps the prediction in an interval whose coverage is assured without any assumption about the model, and the size of the calibration set sets how tight that guarantee is.
Sources: Vovk, Gammerman and Shafer, Algorithmic Learning in a Random World (2005); Angelopoulos and Bates, Conformal prediction: a gentle introduction, Foundations and Trends in Machine Learning (2023); Guo, Pleiss, Sun and Weinberger, On calibration of modern neural networks, ICML (2017); Gama et al., A survey on concept drift adaptation, ACM Computing Surveys (2014); Rudin, Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nature Machine Intelligence (2019).
05At a glance
At a glance, hub and courses
- Build a twin of any placeTwin builderWorked example: the central Apennines: L'Aquila, Amatrice and Norcia · 4 generators · 7 layers · 4 instruments
Igrafu ye-3D
IFakhalthi yeNzululwazi yoMngcipheko neeNdlela
265 nodes1,691 relations9 views
06Ikhonsoli
How AI and machine learning for risk analysis is taught, as a process you can run
Press Play to run this process from this page's own data.
I-AI nokufunda koomatshini kuhlalutyo lomngcipheko
How AI and machine learning for risk analysis is taught
5 subjects in scope, 3 key terms, courses in development, 1 lab, in the Quantitative Risk Analysis department of Risk Science and Methods: from the scope to the method and its terms, the courses that teach it, the labs on the twin, the assessment and the record.
-
01 Scope
- 01
Supervised learning for…
Start · the discipline
Supervised learning for risk outcomes
- 02
Emulators and…
Start · the discipline
Emulators and surrogates of physical and catastrophe models
- 03
Uncertainty…
Start · the discipline
Uncertainty quantification in machine learning
- 04
Conformal prediction…
Start · the discipline
Conformal prediction and distribution-free coverage
- 05
Anomaly detection and…
Start · the discipline
Anomaly detection and change detection
- 01
-
02 Method and terms
- 06
From a trained model to…
Model · the learner
A learned model gives a point prediction with no promise attached; holding out a calibration set, ranking its residuals and taking a…
- 07
Emulator
Record · the discipline
A fast statistical or machine-learning model trained to reproduce the outputs of a slower physical or catastrophe model.
- 09
Conformal prediction
Record · the discipline
A method that wraps any model to give prediction sets with a stated coverage under exchangeability.
- 11
Distribution shift
Record · the discipline
A change between the data a model was trained on and the data it is used on, which can invalidate its accuracy.
- 06
-
04 Labs
- 08
Iwele
Exercise · the learner, on the faculty twin
The method runs on the faculty twin and the admitted map layers.
- 08
-
05 Assess
- 10
Module checks
Gate · the course
Each module ends with a short check against its outcomes.
- 12
Assessed work
Decision · the learner
The assessed work that closes a course: a lab record, a case memo or an assignment, reviewed against the rubric.
- 10
-
06 Record
- 13
Irekhodi yokugqiba
Outcome · the learner
The record stacks into the discipline achievement.
- 14
Discipline achievement
Record · the Academy
The discipline achievement follows the discipline rule, read from the records.
- 13
AI and machine learning for risk analysis · Quantitative Risk Analysis An outline of how the work is designed to run; it does not report the status of any decision or record.
07Ukufunda
Achievements, where this is practised, practised in and across the ecosystem
Achievements
Achievements in this discipline
Course achievements
One for each course with this as its primary discipline, earned when you pass the course's assessed activities.
Department: Quantitative Risk Analysis
Earn two discipline achievements in the Department of Quantitative Risk Analysis. In development.
An achievement is a record of the learning completed and assessed here. How recognition works.
Where this is practised
Where the subject is practised
A field platform page on this subject is in development.
Practised in
The two Foundry systems where it is practised
- FoundryNXS-LAB systemMachine-learning methods are compared under transparent tests
- NXS-NXQ systemModels have a lifecycle of fitness, validation and drift
Foundry system pages describe what each system and its rooms are designed to do; the courses teach the methods these systems apply.
Across the ecosystem
Across the ecosystem
The science, platforms and standards this discipline draws on and contributes to across the ecosystem.
Foundations in the STEM Commons
- The Global Centre for Risk and Innovation (GCRI Canada)Artificial intelligence and machine learningArtificial intelligence and machine learning develop algorithms that learn patterns from data to classify, predict or generate outputs.
- Uncertainty quantification and model validationUncertainty quantification and model validation characterise how uncertainty in inputs, parameters and model structure affects predictions.
- Causal inference and impact evaluationCausal inference and impact evaluation develop methods to determine whether and how interventions or exposures cause outcomes.
Pages of the ecosystem, cited so learners can follow the field into research, platforms, standards and national practice.
08Thatha inxaxheba
Take part in Risk Science and Methods
Qala ngokuba ungubani.
Routes
Indlela
Amaziko esizwe
Ministries, statistics offices and agencies: a national desk and chapter, and a seat in the working groups. National chapters
Indlela
Sectors and professions
Operators, regulators, banks, insurers and professional bodies whose work touches risk science and methods. For organisations
Indlela
Ukufunda
Courses and practice, with a record of every course completed. Free to study. Open learning
Indlela
Imigangatho
Draft, review and adopt the methods and data standards with the working groups. Working groups
Indlela
Communities and research
Universities, civil society and individual experts: projects, data and review. Write to the Academy
Open calls
| Open call | Seeks | Closes |
|---|---|---|
| Founding call for experts to the working groups CALL-FOUNDING-EXPERTS | expert, editor, chair | 2026-12-15 |
| Open nomination for the three independent expert seats of the Global Standards Council CALL-GSC-IND | ilungu-lebhunga | 2026-12-01 |
| Nominations of interim regional members (applies only if the transitional provision is decided) CALL-GSC-REG | ilungu-lebhunga | 2026-12-15 |
| Uphononongo loshwankathelo olungundoqo lomngcipheko lwegagasi-1 (C1, C2, C3, C4, C8, C10, C11) CALL-REVIEW-WAVE1 | contributor | 2027-01-31 |
| Iingcali zemigangatho engundoqo yenzululwazi yomngcipheko: ukungaqiniseki nokuhlelwa kobungqina, izikali, isigwebo seengcali esicwangcisiweyo, ukunyamezeleka, iziganeko ezigqithisileyo nezidibeneyo, umngcipheko weentlekele nowobukho, iinqobo ezisesikweni nokulingana, iiyunithi nomlinganiselo (NRS-C17 ukuya ku-NRS-C24) CALL-CORE-SCIENCE | expert, editor, chair | 2027-01-31 |
| Independent reviewers for the families of the hazard atlas, three from at least two regions for each family (NHS2-R19), priority families first CALL-ATLAS-REVIEW | ingcali | 2027-03-31 |
| Amalungu ePhaneli yethutyana yoPhononongo loKhuseleko lweTekhnoloji eNtsha nabaphononongi beqela elibomvu (ukhuseleko lwebhayoloji, ukhuseleko lwe-AI, ukhuseleko lweziseko ezibalulekileyo); izicelo ziphathwa ngemfihlo CALL-FRONTIER-SAFETY-REVIEW | ingcali | 2026-11-30 |
| Iingcali zeontoloji, inzululwazi yemilinganiselo, ukuqinisekiswa kwesoftware novavanyo lokuhambelana zerejista yeemfuno, umfanekiso weNHS-2 woomatshini, iiseti zovavanyo lokuhambelana nesikimu sovavanyo lokuhambelana CALL-CONFORMANCE-ONTOLOGY | expert, editor | 2027-01-31 |
Calls past their closing date are hidden automatically. Every call, proposal and route: open areas and calls.
Cells and CERTs
Gather the cell
Bring together three to seven people with complementary skills: data, field practice, policy and communication.
Register it
Write to the team with the cell's name, members, area and region. You receive a reference and a contact. Write to the Academy
Choose a charter
Pick one focus: research and evidence, data and maps, standards review, or community preparedness.
Learn together
Follow shared study paths at Risk Academy, with practice and a record of every course completed. Hazards and resilience
Lead and report
Hold regular sessions. Publish what the cell learns through Reports. Bring findings to the working groups. Working groups
Specialise as a Community Emergency Response Team (CERT)
A CERT prepares its neighbourhood for the hazards it faces. It builds a local hazard profile from this site's map. It trains in household and street preparedness. It learns first aid and light search through recognised local providers, and it runs drills. A CERT works alongside the local emergency services and follows their direction. It never replaces them and is not an emergency service.
Isalathiso
What a national team or a local community can do on The Global Risks Index (NXS-GRX) today.
NXS-GRX
Your country's profile
Open risk profiles for 195 countries, with sources and licences on every value. Open the Index
NXS-GRX
Correct or reply
Correct a value or reply for your country. The change is logged with its effect. Open the Index
NXS-GRX
Offer data
Offer a dataset. It enters through the licence gate, with your credit. Open the Index
NXS-GRX
Use the open data
Layers, profiles and tiles through the open API. Open the Index
09References
Core references and where to begin
Start with Hastie, then Guo, then Angelopoulos.
- Hastie, T., Tibshirani, R. and Friedman, J., The Elements of Statistical Learning, second edition, Springer (2009). DOI
- Kennedy, M. C. and O'Hagan, A., Bayesian calibration of computer models, Journal of the Royal Statistical Society Series B (2001). DOI
- Vovk, V., Gammerman, A. and Shafer, G., Algorithmic Learning in a Random World, Springer (2005). DOI
- Angelopoulos, A. N. and Bates, S., Conformal prediction: a gentle introduction, Foundations and Trends in Machine Learning (2023). DOI
- Guo, C., Pleiss, G., Sun, Y. and Weinberger, K. Q., On calibration of modern neural networks, Proceedings of the 34th International Conference on Machine Learning (2017). proceedings.mlr.press
- Lakshminarayanan, B., Pritzel, A. and Blundell, C., Simple and scalable predictive uncertainty estimation using deep ensembles, Advances in Neural Information Processing Systems 30 (2017). arxiv.org
- Rudin, C., Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nature Machine Intelligence (2019). DOI
- Reichstein, M. et al., Deep learning and process understanding for data-driven Earth system science, Nature (2019). DOI
- Pearl, J., Causality: Models, Reasoning, and Inference, second edition, Cambridge University Press (2009). DOI
- Barocas, S., Hardt, M. and Narayanan, A., Fairness and Machine Learning: Limitations and Opportunities, MIT Press (2023). fairmlbook.org
10Imibuzo
Imibuzo, iimpendulo namagama aphambili
Ingaba ukufunda apha kusimahla?
Ewe. Yonke ikhosi nezixhobo zokufunda ze-Risk Academy, ekhokelwa yi-Nexus Consortium, zisimahla ukuzifunda, ngesantya sakho okanye neqela. Ukuthatha inxaxheba okuqhelekileyo akunamrhumo; umsebenzi ohlawulwayo, ukuqeqeshelwa umsebenzi neefelowshiphu ngamalungiselelo ahlukileyo.
Ukugqiba ikhosi kundinika ntoni?
A completion record: the course, its version, the date and what was assessed, private by default and shared by you. It stacks into discipline, department and programme achievements. A record states what was studied and assessed and does not certify competence; it is not a licence, a professional title or an employment status.
Idatha neendawo zivela phi?
Every faculty teaches on a twin of a real place built from open data; each layer carries its provider, licence and date, and every result on the Academy states its source. Maps and figures are indicators for teaching, never an assessment of a place.
How do I take part beyond the courses in aI and machine learning for risk analysis?
Form a competence cell of three to seven people, study a shared pathway, take a supervised assignment in the National Working Groups, or bring a cohort from your institution. For organisations.
Amagama aphambili
- Emulator
- A fast statistical or machine-learning model trained to reproduce the outputs of a slower physical or catastrophe model.
- Conformal prediction
- A method that wraps any model to give prediction sets with a stated coverage under exchangeability.
- Distribution shift
- A change between the data a model was trained on and the data it is used on, which can invalidate its accuracy.
- Calibrated classifier
- A classifier whose predicted probabilities match observed frequencies.
IingxeloYiba negalelo
Faka ubungqina obuvela kweli phepha.
Reports is the Consortium's quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
Risk Academy
- EmailGateway GW-ACADEMY Send it from this page Evidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen deposit Deposit it and submit to the community Publish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsRisk Academy See every way to contribute The contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
I-AI nokufunda koomatshini kuhlalutyo lomngcipheko
Prepare with a related course
Open courses in connected disciplines teach the methods this discipline will build on.
Khangela ikhathalogu yeekhosiBack to Department of Quantitative Risk Analysis