Illustrative: a joint probability density whose dependence shifts over time
Risk Academy, led by the Nexus Consortium · Risk Science and Methods · Department of Decision, Behaviour and Communication
Turning expert knowledge into a reviewable distribution
Expert judgement and elicitation
Where data are thin, risk still has to be quantified, and expert judgement is the method of record.
- Courses
- In preparation
- Levels
- In development
- Labs
- In development
- Structured expert judgement protocols
- The Classical Model and performance-based weighting
- IDEA and Delphi protocols
- SHELF and behavioural aggregation
- Seed questions and expert calibration
- Heuristics and biases in judgement
- Eliciting probability distributions
- Documenting and reviewing an elicitation
- Expert judgement in catastrophe and climate assessment
Build a twin of any place
Search any place or open the world; the layers and generators follow expert judgement and elicitation on the planet map, the globe and in 3D.
Risk Science and MethodsAny placeTwin builder
- Terrain 24 to 2,805 m
- 3 reference earthquakes, M6.2 to M6.6
- Earthquakes M4.5 and above, past week: read live when the 3D twin opens
- USGS ShakeMap intensity contours: read live when the 3D twin opens
Drag to move, right drag or Shift with a drag to turn and tilt, Ctrl with the wheel to zoom. Keyboard: arrows move, Shift with arrows turn and tilt, + and – zoom, 0 resets, V steps through the views, Escape clears.
Places4
Layers4
iCredits
- Terrain Surface height in whole metres on the plan's grid of cells about 455 by 605 metres; the box rises from 24 m in the Tiber valley to 2,805 m on the Gran Sasso. Produced using Copernicus WorldDEM-30 © DLR e.V. 2010–2014 and © Airbus Defence and Space GmbH 2014–2018 provided under COPERNICUS by the European Union and ESA; all rights reserved. read 2026-10-17. Estimate: A digital surface model: trees and roofs are in the heights. Relief exaggeration is printed on the stage.
- Reference earthquakes The three reference main shocks (L'Aquila 2009, Amatrice 2016, Norcia 2016): origin, magnitude, depth, time and review status from the USGS detail record, the hypocentre drawn below the ground. U.S. Geological Survey Earthquake Hazards Program. read 2026-10-17. Estimate: Reported solutions as reviewed by the USGS; magnitudes and depths carry the USGS stated uncertainties, which the event page links rather than restates.
- Earthquakes M4.5 and above, past week (live) Reviewed and automatic seismic solutions of the past week inside the box, with magnitude and depth, read from the world feed and filtered to the box in your browser; the distribution fitter reads the same world feed for its sample. U.S. Geological Survey; reported detections, automatic solutions preliminary; never a warning. Live: this page requests earthquake.usgs.gov when the layer is opened; the data date is printed with it. Estimate: Reported detections; automatic solutions are preliminary; never a warning.
- USGS ShakeMap intensity contours The ShakeMap MMI contours of each reference event draped on the terrain, read from the cont_mmi.json file the event's detail record names: what one realisation of the sampler's field looked like. U.S. Geological Survey ShakeMap and U.S. Geological Survey PAGER (preliminary, automatically generated; see the USGS disclaimers), with the product version and time. Live: this page requests earthquake.usgs.gov when the layer is opened; the data date is printed with it. Estimate: Preliminary and automatically generated; PAGER levels are USGS estimates with uncertainty, never a warning or a loss figure of the Academy.
Why this place: One box of about 82 by 77 kilometres holds the 2009 L'Aquila (M6.3), 2016 Amatrice (M6.2) and Norcia (M6.6) earthquakes, the Gran Sasso massif and the normal-fault basins between them. It is the reference case for every method the faculty teaches: magnitude-frequency fitting on a dense catalogue, return periods, probabilistic hazard, model validation, and risk communication after the L'Aquila trial of 2012, which turned on how probabilities were communicated. Data quality: Copernicus DEM GLO-30 covers the relief in two tiles, the USGS detail records of the three main shocks are reviewed, and the live feed is open. It connects to the Quantitative risk analysis and statistics, Catastrophe, exposure and vulnerability modelling, Uncertainty and decision making, Risk communication and risk literacy and Geospatial analysis, Earth observation and GeoAI disciplines and to the spatial intelligence pages of the ecosystem sites.
Flat plan of the same data: relief, the three reference epicentres with magnitude and depth, and the numbers below; the sampler's figure shows one precomputed field with its parameters. The 3D twin opens on devices that can draw it, once its data and code have been checked.
The boundaries, names and designations shown do not imply any judgement on the legal status of any territory, or any endorsement or acceptance of such boundaries. Countries: Natural Earth (public domain).
Hazards and exposure in Italy
Flood water, active fires and surface heat beside population and night-time lights, over Italy.
The interactive map needs JavaScript; the layer list and sources are below.
Layers 3
Hazards and exposure
Keyboard, with the map focused
Hold Ctrl or ⌘ and scroll to zoom
Latest available
NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth
Loading the latest NASA imagery.
Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.
Hazards and exposure, framed on Italy.Satellite and model products as NASA publishes them, each with its own date. They are indicators, not assessments of any place.
Data: NASA GIBS · ASTER GDEM · Blue Marble · © OpenStreetMap contributors · Natural Earth · NASA EONET · U.S. Geological SurveySources and licences
- We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
- We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
- VIIRS near-real-time products (flood, land surface temperature, snow cover, corrected reflectance), NASA LANCE; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
- NASA Fire Information for Resource Management System (FIRMS), VIIRS 375 m active fire product, NASA LANCE; vector tiles via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
- Center for International Earth Science Information Network (CIESIN), Columbia University. 2018. Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11. NASA Socioeconomic Data and Applications Center (SEDAC). Licensed under CC BY 4.0. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). doi:10.7927/H49C6VHW. CC BY 4.0
- VIIRS Black Marble night-time lights, NASA Goddard Space Flight Center; distributed by the NASA Level-1 and Atmosphere Archive and Distribution System (LAADS DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
- MODIS and VIIRS atmosphere products, NASA LANCE and LAADS DAAC; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
- ASTER GDEM is a product of METI and NASA (ASTER Global Digital Elevation Model V003, NASA LP DAAC); imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). Cite as: NASA/METI/AIST/Japan Spacesystems and U.S./Japan ASTER Science Team (2019). ASTER Global Digital Elevation Model V003. doi:10.5067/ASTER/ASTGTM.003. NASA open data
- Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com. Public domain
- © OpenStreetMap contributors (ODbL), via NASA GIBS (coastlines, boundaries, roads and place names). Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). ODbL
- Blue Marble, NASA Earth Observatory; imagery via NASA GIBS. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). NASA open data
- Context on the globe and flat map: coastlines, boundaries, roads and place names from NASA GIBS reference layers (OpenStreetMap data, ODbL, © OpenStreetMap contributors); Blue Marble imagery as an optional base; 3D terrain and the elevation read-out from the ASTER GDEM elevation colour index (5 m to 200 m classes, exaggerated for legibility). Boundaries are drawn as published and imply no position on any territory.
- Globe: MapLibre GL JS 6.9.1, BSD-3-Clause, © MapLibre contributors.
- Live events: NASA Earth Observatory Natural Event Tracker (EONET); events are indicative and keep their original source links.
- Live: NASA EONET Events are indicative, not authority warnings.
- Live: U.S. Geological Survey Events are indicative, not authority warnings.
- NASA GIBS: We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS).
- River discharge forecast, 30 days with range (GloFAS via Open-Meteo): Contains modified Copernicus Emergency Management Service information 2026; not a flood warning.
- Weather data by Open-Meteo.com (https://open-meteo.com/), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).
- Optional layers (10 sources in the layer list: live, forecast, observed and context) print their provider credit on the map and here when switched on; forecasts are model output and derived indices are estimates.
Layers and dates
- Flood water, 3-day composite: VIIRS Global Flood Product, NASA LANCE · 14 Oct 2026
- Active fires and thermal anomalies: VIIRS NOAA-20 375 m (FIRMS), NASA LANCE FIRMS · 13 Oct 2026
- Population density, 2020: Population: CIESIN, Columbia University, Gridded Population of the World v4.11 (2018), NASA SEDAC, CC BY 4.0 · static
- Land surface temperature, day, 8-day composite: VIIRS NOAA-20 Level 3, 8-day, NASA LANCE · 14 Sep 2026
- Night-time lights, daily: VIIRS Black Marble (VNP46A2), NASA LAADS DAAC · 12 Oct 2026
- Aerosol optical depth, daily: VIIRS NOAA-20 Deep Blue AOD, NASA LANCE · 13 Oct 2026
The subject, methods and measures and debates and bodies
The field and its scope
For analysts, assessment authors and decision makers who must estimate uncertain quantities from experts when data are thin, and do it in a way that others can review.
Quantities and units
Methods and their limits
Each method is listed with what it assumes and where it fails.
What the field debates
Bodies that set practice, and the data they publish
Bodies and standards are named so learners can find and apply their published work; the Academy teaches their use and links to the publisher, and no affiliation is implied.
From an expert's quantiles to a weight: seed questions, calibration, information and the aggregate that results
At a glance, hub and courses
Expert judgement and elicitation at a glance
risksacademy.comrisk-sciencediscipline
- Open, 0
- In preparation, 0
- Subtopics, 9
Research and tools
Research and preprints as they are published, by discipline, with the faculty's other tools.
The hub lists the newest research and preprints for Expert judgement and elicitation, each with its source and date. Open the research hub
Courses in this discipline
Courses for this discipline are in development; until they open, study it through the connected courses and open lessons below.
The course catalogue lists every open course of the Academy.
How Expert judgement and elicitation is taught, as a process you can run
Press Play to run this process from this page's own data.
Achievements, where this is practised, practised in and across the ecosystem
Achievements in this discipline
Course achievements
One for each course with this as its primary discipline, earned when you pass the course's assessed activities.
Department: Decision, Behaviour and Communication
Earn two discipline achievements in the Department of Decision, Behaviour and Communication. In development.
An achievement is a record of the learning completed and assessed here. How recognition works.
Where the subject is practised
A field platform page on this subject is in development.
The two Foundry systems where it is practised
- FoundryNXS-DSS systemExpert estimates enter decisions as stated distributionsDecisions room
- NXS-CMD systemPanels of experts deliberate and decide in councilCouncil room
Foundry system pages describe what each system and its rooms are designed to do; the courses teach the methods these systems apply.
Across the ecosystem
Take part in Risk Science and Methods
Start from who you are.
Core references and where to begin
Start with Aspinall, then Morgan, then O'Hagan.
Questions, answers and key terms
Is study here free?
What does completing a course give me?
Where do the data and the places come from?
How do I take part beyond the courses in expert judgement and elicitation?
Key terms
Structured expert judgement
A documented protocol for eliciting and combining expert estimates so that the result can be reviewed.
Classical Model
Cooke's method that weights experts by their calibration and informativeness on seed questions with known answers.
Seed question
A question whose answer is known to the analyst but not the experts, used to score expert performance.
Delphi method
An iterative, anonymous survey of experts with controlled feedback between rounds.
Contribute evidence from this page.
Reports is the Consortium's quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
Around Expert judgement and elicitation
Prepare with a related course
Open courses in connected disciplines teach the methods this discipline will build on.