Illustrative: an ensemble of trajectories, its quantile fan and the outcome as it is observed
Risk Academy, led by the Nexus Consortium · Risk Science and Methods · Department of Quantitative Risk Analysis
Forecasts as distributions, judged by proper scoring rules
Probabilistic forecasting and verification
A risk number is a forecast, and a forecast is only as good as its verification.
- Department
- Department of Quantitative Risk Analysis
- Courses
- In preparation
- Levels
- In development
- Labs
- In development
- Probabilistic and ensemble forecasts
- Proper scoring rules (Brier, CRPS, logarithmic)
- Calibration, sharpness and reliability diagrams
- Discrimination, ROC curves and hit and false alarm rates
- Verification of categorical warnings and triggers
- Forecast value and the cost-loss model
- Post-processing and statistical calibration of ensembles
- Machine-learning weather and hazard forecasts and their verification
- Forecast combination and model averaging
Build a twin of any place
Search any place or open the world; the layers and generators follow probabilistic forecasting and verification on the planet map, the globe and in 3D.
- Terrain 24 to 2,805 m
- 3 reference earthquakes, M6.2 to M6.6
- Earthquakes M4.5 and above, past week: read live when the 3D twin opens
- USGS ShakeMap intensity contours: read live when the 3D twin opens
Drag to move, right drag or Shift with a drag to turn and tilt, Ctrl with the wheel to zoom. Keyboard: arrows move, Shift with arrows turn and tilt, + and – zoom, 0 resets, V steps through the views, Escape clears.
Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices that can draw it, once its data and code have been checked.
The boundaries, names and designations shown do not imply any judgement on the legal status of any territory, or any endorsement or acceptance of such boundaries. Countries: Natural Earth (public domain).
The subject, methods and measures and debates and bodies
The field and its scope
For forecasters, early warning analysts, modellers and the people who act on forecasts, who need to issue risk as a calibrated distribution and to judge forecasts fairly against what happened.
Quantities and units
Methods and their limits
Each method is listed with what it assumes and where it fails.
What the field debates
Bodies that set practice, and the data they publish
Bodies and standards are named so that learners can find and apply their published work. The Academy teaches the use of that work and links to the publisher. No affiliation is implied.
From a probability forecast to its value for a user: hits, false alarms and the cost-loss ratio that decides
At a glance, hub, what is changing now and courses
Probabilistic forecasting and verification at a glance
risksacademy.comrisk-sciencediscipline
- Open, 0
- In preparation, 0
- Subtopics, 9
Research and tools
Research and preprints as they are published, by discipline, with the faculty's other tools.
The hub lists the newest research and preprints for Probabilistic forecasting and verification, each with its source and date. Open the research hub
What is changing now
Recent developments in probabilistic forecasting and verification, each with its source; sources checked on 16 October 2026.
- Technology
ECMWF made its machine-learning ensemble forecast operational
ECMWF's AIFS ENS became operational, producing 51-member ensemble forecasts alongside the physics-based system. It outperforms physics-based models on many measures, runs over ten times faster with far less energy, but still relies on physics-based data assimilation.
Why it matters. Hybrid physical and machine-learning ensembles change how forecast uncertainty is produced and must be validated.
Source: European Centre for Medium-Range Weather Forecasts, ECMWF's ensemble AI forecasts become operational, 1 July 2025.
Courses in this discipline
Courses for this discipline are in development. Until they open, study the discipline through the connected courses and open lessons below.
The course catalogue lists every open course of the Academy.
How Probabilistic forecasting and verification is taught, as a process you can run
Press Play to run this process from this page's own data.
Process explorer
How Probabilistic forecasting and verification is taught
14 steps5 stages
Achievements, where this is practised, practised in and across the ecosystem
Achievements in this discipline
Course achievements
One for each course with this as its primary discipline, earned when you pass the course's assessed activities.
Department: Quantitative Risk Analysis
Earn two discipline achievements in the Department of Quantitative Risk Analysis. In development.
An achievement is a record of the learning completed and assessed here. See how recognition works.
Where the subject is practised
A field platform page on this subject is in development.
The two Foundry systems where it is practised
- FoundryNXS-EWS systemForecasts and warnings are judged by how they verifyForecast Verification room
- NXS-LAB systemCompeting forecast methods are compared under transparent testsBenchmarks room
Foundry system pages describe what each system and its rooms are designed to do; the courses teach the methods these systems apply.
Across the ecosystem
Take part in Risk Science and Methods
Start from who you are.
Core references and where to begin
Start with Murphy, then Gneiting, then Brier.
Questions, answers and key terms
Is study here free?
What does completing a course give me?
Where do the data and the places come from?
How do I take part beyond the courses in probabilistic forecasting and verification?
Key terms
Proper scoring rule
A score that is optimised in expectation only when the forecaster reports their true probability, such as the Brier score or CRPS.
Calibration
Agreement between forecast probabilities and observed frequencies: events forecast at 30 per cent happen about that often.
Sharpness
How concentrated a forecast distribution is, judged only once calibration holds.
Continuous ranked probability score
A proper score for a forecast distribution of a continuous quantity, in the units of the quantity.
Contribute evidence from this page.
Reports is the Consortium's quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
Around Probabilistic forecasting and verification
Prepare with a related course
Open courses in connected disciplines teach the methods this discipline will build on.