Risk Academy, a Nexortium programmeRisk Science and MethodsDepartment of Quantitative Risk Analysis

Forecasts as distributions, judged by proper scoring rules

Probabilistic forecasting and verification

A risk number is a forecast, and a forecast is only as good as its verification.

risksacademy.comrisk-sciencediscipline

Probabilistic forecasting and verification, drawn as uncertainty bands: 0 primary courses open, 0 in preparation, 0 approved for authoring and 0 also taught here, across 9 subtopics. Courses: none yet.L1L2L3L420
Probabilistic forecasting and verification at a glance. Drawn as uncertainty bands.
  • Open, 0
  • In preparation, 0
  • Subtopics, 9
Course codes from the curriculum; study levels L1 to L4.
Twin builder and simulatorThe central Apennines: L'Aquila, Amatrice and Norcia4 twin generators, 8 layers, 4 instruments
courses open now
0
in preparation
0
subtopics in its scope
9
reference earthquakes, M6.2 to M6.6
3

Twin figures for the central Apennines: L'Aquila, Amatrice and Norcia, counted from its layers at their stated dates.

Twin builder and simulator.

For Probabilistic forecasting and verification: the faculty's twin of L'Aquila on the planet map, the globe and in 3D. Pick a generator, switch layers, run the instruments; every layer prints its provider and date.

The central Apennines: L'Aquila, Amatrice and Norcia: a flat plan of the faculty twin. Terrain 24 to 2,805 m; 3 reference earthquakes, M6.2 to M6.6; Earthquakes M4.5 and above, past week (live): live, loads in the twin; USGS ShakeMap intensity contours: live, loads in the twin. Contours every 500 metres from the elevation grid; reference earthquakes as marks.
The central Apennines: L'Aquila, Amatrice and Norcia, Italy. Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices with WebGL once its data and library have been checked.
  • Terrain 24 to 2,805 m
  • 3 reference earthquakes, M6.2 to M6.6
  • Earthquakes M4.5 and above, past week (live): live, loads in the twin
  • USGS ShakeMap intensity contours: live, loads in the twin
Credits and dates per layer are listed under the stage. Relief in the twin is exaggerated ×1.5 for legibility.

Drag to move, right drag or Shift with a drag to turn and tilt, Ctrl with the wheel to zoom. Keyboard: arrows move, Shift with arrows turn and tilt, + and - zoom, 0 resets, V steps through the views, Escape clears.

Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices with WebGL once its data and library have been checked.

The map loads from this site as it nears the view (MapLibre 6.9.1, checked by hash).

The discipline in brief.

The discipline teaches probabilistic and ensemble forecasts, the proper scoring rules that judge them (the Brier score, the continuous ranked probability score and the logarithmic score), calibration and sharpness read from reliability diagrams, hit and false alarm rates for warnings and triggers, and forecast value through the cost-loss model. Machine-learning forecasts are part of the subject now: ECMWF has run its AIFS Single model in operations since February 2025 and its AIFS ensemble since July 2025, and both are verified with the same scores as the physics-based system. A graduate can score a forecast set, read its reliability and say whether acting on it pays.

This area covers how risk is forecast as a calibrated probability distribution rather than a single value, and how forecasts are judged against what happened with proper scoring rules. It is taught in the department of Quantitative Risk Analysis, Faculty of Risk Science and Methods, through courses still to be authored.

What the discipline covers.

9 subtopics, taught across its courses and labs.

Courses
0 open, 0 in preparation
Levels
None yet
Labs
Named when the courses are authored
  • Probabilistic and ensemble forecasts
  • Proper scoring rules (Brier, CRPS, logarithmic)
  • Calibration, sharpness and reliability diagrams
  • Discrimination, ROC curves and hit and false alarm rates
  • Verification of categorical warnings and triggers
  • Forecast value and the cost-loss model
  • Post-processing and statistical calibration of ensembles
  • Machine-learning weather and hazard forecasts and their verification
  • Forecast combination and model averaging

What is changing now.

Recent developments in probabilistic forecasting and verification, each with its source; sources checked on 16 October 2026. Summaries are short readings of the source; follow the link for the full account.

  1. Technology

    ECMWF made its machine-learning ensemble forecast operational

    ECMWF's AIFS ENS became operational, producing 51-member ensemble forecasts alongside the physics-based system. It outperforms physics-based models on many measures, runs over ten times faster with far less energy, but still relies on physics-based data assimilation.

    Why it matters. Hybrid physical and machine-learning ensembles change how forecast uncertainty is produced and must be validated.

    Source: European Centre for Medium-Range Weather Forecasts, ECMWF's ensemble AI forecasts become operational, 1 July 2025.

The faculty hub.

News and research as they are published, per discipline, with the faculty's other tools.

The newest research, preprints, news and humanitarian reports for Probabilistic forecasting and verification, each with its source and date, load here. Open the research and news hub

No course yet.

No course has this as its primary discipline yet. It is listed so the gap stays visible.

Courses also taught here and the connected disciplines teach the methods it will build on.

What you can earn here.

  1. Course achievements

    One for each of the 0 courses with this primary discipline, when you pass its assessed activities; 0 can be earned now.

  2. Department: Quantitative Risk Analysis

    Earn two discipline achievements in the Department of Quantitative Risk Analysis. In preparation.

Achievements record learning completed here; they do not certify competence. How recognition works.

No ecosystem page yet.

No ecosystem page covers this subject yet. A page has been requested; no loosely related page is cited in its place.

The two Foundry systems where it is practised.

  • Nexus FoundryEWS systemForecasts and warnings are judged by how they verifyForecast Verification room
  • Labs systemCompeting forecast methods are compared under transparent testsBenchmarks room

Nexus Foundry system pages describe what each system and its rooms are designed to do; they are not a service on offer, and studying here gives no access to them.

Terms used on this page.

Probabilistic forecasting and verification

Prepare with a related course.

Open courses in connected disciplines teach the methods this discipline will build on.

Skip to toolbar