Risk Academy, led by the Nexus Consortium · Governance, Policy and International Affairs · Department of Science, Technology and Society
Evidence synthesis, integrity, reproducibility, grading and corrections
Ucwaningo, ubufakazi nokuhlangana kwesayensi nenqubomgomo
Research, evidence and the science-policy interface covers how knowledge about risk is produced, reviewed, recorded and published so that it can be trusted, reused and corrected, and how it reaches decisions.
Map research, evidence and the science-policy interface on a real placeOpen RAF-103
Illustrative: branching futures, each branch weighted by its probability
01Ububanzi
What the discipline covers
The subtopics its courses and labs teach.
- Izifundo
- In preparation
- Amazinga
- In development
- Amalabhorethri
- Trigger lab
- The science-policy interface
- Evidence synthesis
- Ubuqotho bocwaningo
- Reproducibility
- Knowledge translation
- Peer review and publication
- Evidence grading
- Corrections and versioning
Courses in development extend the scope to one further subject: Research methods and reproducible evidence for risk.
02Amamephu
Build a twin of any place
Search any place or open the world; the layers and generators follow research, evidence and the science-policy interface on the planet map, the globe and in 3D.
- 13 countries
- First-level divisions, geoBoundaries gbOpen with a licence gate per boundary: read live when the 3D twin opens
- Natural events, NASA EONET: read live when the 3D twin opens
Drag to turn the globe, Ctrl with the wheel to zoom. Keyboard: arrows turn, + and – zoom, 0 resets, V steps through the views, Escape clears.
Flat map of the same data: the 13 countries on a graticule and the numbers below. The globe opens on devices that can draw it, once its code has been checked; the live events and divisions load in the globe on request, each with its data date.
The boundaries, names and designations shown do not imply any judgement on the legal status of any territory, or any endorsement or acceptance of such boundaries. Countries: Natural Earth (public domain).
03The subject
The subject, methods and measures and debates and bodies
The discipline
The field and its scope
For researchers, evidence synthesists, science advisers and the analysts who use their work, who need to judge how knowledge about risk is produced, graded, corrected and brought to decisions.
This field studies how knowledge about risk is produced, reviewed, recorded, published and corrected so that it can be trusted and reused, and how it reaches decisions. It covers evidence synthesis and systematic review, research integrity and the handling of misconduct, reproducibility and open data, peer review and publication, the grading of evidence by certainty, knowledge translation into policy and practice, and the practice of corrections, retractions and versioning. It sits at the science-policy interface described by Pielke, Jasanoff and Gluckman, where advisers must be useful to decision makers without surrendering independence.
The field's working materials are reporting standards such as PRISMA, grading systems such as GRADE, the FAIR principles for data, codes of research conduct such as the ALLEA code, and the infrastructures of registration, preprints and retraction databases. Its evidence base includes the replication projects that found many published findings do not reproduce and the meta-research that explains why. It serves science advisers and their offices, research funders and institutions, journals, public health and regulatory agencies, and the organisations that commission or rely on evidence for risk decisions. Its practical question is how to state what is known, how certain it is and what would change the conclusion, in a form a decision maker can use.
Measures
Quantities and units
| Quantity | Unit or scale | What to know |
|---|---|---|
| Replication rate | share of studies whose replication reaches the same conclusion | The Open Science Collaboration (2015) found that 36 per cent of 100 replicated psychology studies produced significant results in the same direction; rates differ by field and by the definition of success, and selection of studies for replication is not random. |
| Retractions | count per year, and rate per 10,000 publications | The Retraction Watch database, now held by Crossref, recorded more than 10,000 retractions in 2023; the count reflects detection and publisher practice as much as the underlying rate of error or misconduct. |
| Certainty of evidence | four ordinal levels: high, moderate, low, very low | GRADE assigns certainty to a body of evidence for each outcome by considering risk of bias, inconsistency, indirectness, imprecision and publication bias; judgements are structured but not mechanical, and two panels can grade the same evidence differently. |
| Time from evidence to guideline | months or years between a pivotal study and its uptake in formal guidance | Estimated in studies of guideline updating and in living review programmes; the lag depends on the guidance body's cycle and is shortest where living synthesis has been adopted. |
| Data and code availability | share of publications with accessible data and code | Audits of journals report shares that vary from a few per cent to a majority depending on field and policy; stated availability often exceeds actual availability on request. |
Izindlela
Methods and their limits
Each method is listed with what it assumes and where it fails.
| Indlela | Assumes | Fails when |
|---|---|---|
| Systematic review and meta-analysis | That a transparent, pre-registered search and selection protocol, reported after PRISMA, reduces bias in summarising a body of evidence and that effect sizes can be pooled when studies are sufficiently similar. | When publication bias has shaped the literature, when studies are too heterogeneous to pool, when the question is poorly framed, or when the review is out of date before publication. |
| GRADE evidence grading | That certainty in a body of evidence can be judged against explicit criteria and communicated in a small number of levels that decision makers understand. | When it is applied outside the clinical settings it was designed for without adaptation, when it downgrades all observational evidence regardless of design, or when the levels are read as probabilities. |
| Replication and reproducibility checks | That independent re-analysis or re-collection tests whether a finding is reliable and that the results feed back into the record. | When replications are underpowered or differ in context, when there is no venue or incentive to publish them, or when the original data were never shared. |
| Science advice brokering | After Pielke (2007), that advisers can present the range of options and their evidence without advocating one, and that this preserves trust across political divides. | When the evidence is uncertain and contested, when the adviser's position is institutionally captured, or when decision makers want a single answer; Funtowicz and Ravetz (1993) describe the post-normal conditions under which standard advice breaks down. |
| Living evidence synthesis | That continual updating of reviews with new studies, supported by automation, keeps guidance current for fast-moving risks. | When the inflow of low-quality studies swamps screening, when funding for maintenance lapses, or when users cannot tell which version they relied on. |
Debates
What the field debates
- Honest broker or advocate
- Pielke (2007) argues that advisers serve best by expanding the choices available and leaving values to decision makers. Critics reply that the broker stance is itself a political choice, that on existential or urgent risks silence on values is abdication, and that the public expects scientists to say what should be done. Case studies of climate and pandemic advice are claimed by both sides.
- Is there a reproducibility crisis
- Ioannidis (2005) argued from statistical first principles that most published findings in low-power, high-flexibility fields are false, and replication projects since have found low rates. Others argue that failed replications are expected in exploratory science, that the projects over-sampled fragile designs, and that the reforms, pre-registration and open data, are already changing practice. Munafò and colleagues (2017) set out the reform programme.
- Open data against privacy and misuse
- Advocates of mandatory sharing cite the FAIR principles, the gains in verification and reuse, and the costs of unshared data in crises. Opponents cite re-identification risk, dual-use hazards, the burden on researchers and the exploitation of data from low-income countries without benefit. Both accept the principle of as open as possible, as closed as necessary, and disagree on where the line falls.
Bodies and data
Bodies that set practice, and the data they publish
Bodies and standards are named so learners can find and apply their published work; the Academy teaches their use and links to the publisher, and no affiliation is implied.
| Body | Role in the field |
|---|---|
| International Science Council | Global membership body of scientific unions and academies that sets positions on science advice, open science and research integrity. |
| International Network for Governmental Science Advice (INGSA) | Network of practitioners of science advice to governments that develops guidance and capacity for the science-policy interface. |
| Cochrane | Produces systematic reviews of health interventions and maintains the methods handbook that other fields adapt. |
| GRADE Working Group | Maintains the GRADE approach to rating certainty of evidence and strength of recommendations. |
| Committee on Publication Ethics (COPE) | Sets guidance for editors and publishers on corrections, retractions and the handling of misconduct allegations. |
| All European Academies (ALLEA) | Publishes the European Code of Conduct for Research Integrity, the reference for funders and institutions in Europe. |
| Dataset | Steward |
|---|---|
| Retraction Watch Database | Crossref and the Center for Scientific Integrity |
| OpenAlex | OpenAlex (OurResearch) |
| Cochrane Library | Cochrane |
| PROSPERO register of systematic reviews | Centre for Reviews and Dissemination, University of York |
04Indlela
From a prior to a published claim: why a significant finding is often false, and what that does to advice
The probability that a published positive finding is true depends on the prior odds that the hypothesis was true, the study's power and its false-positive rate; low priors and low power make most significant results wrong, which is why evidence grading reads the body of evidence rather than a single paper.
Sources: Ioannidis, Why Most Published Research Findings Are False (2005); Open Science Collaboration, Estimating the reproducibility of psychological science (2015); Guyatt and others, GRADE: an emerging consensus on rating quality of evidence and strength of recommendations (2008); Pielke, The Honest Broker: Making Sense of Science in Policy and Politics (2007); Munafò and others, A manifesto for reproducible science (2017).
05At a glance
At a glance, standards and evidence, research and evidence, what is changing and learning
- Build a twin of any placeTwin builderWorked example: the Horn of Africa and the Nile basin states · 4 generators · 10 layers · 3 instruments
Igrafu ye-3D
Isikhungo Sokuphatha, Inqubomgomo Nezindaba Zamazwe Omhlaba
299 nodes1,837 relations9 views
06Ikhonsoli
How Research, evidence and the science-policy interface is taught, as a process you can run
Press Play to run this process from this page's own data.
Ucwaningo, ubufakazi nokuhlangana kwesayensi nenqubomgomo
How Research, evidence and the science-policy interface is taught
5 subjects in scope, 3 key terms, courses in development, 1 lab, in the Science, Technology and Society department of Governance, Policy and International Affairs: from the scope to the method and its terms, the courses that teach it, the labs on the twin, the assessment and the record.
-
01 Scope
- 01
The science-policy…
Start · the discipline
The science-policy interface
- 02
Evidence synthesis
Start · the discipline
Evidence synthesis
- 03
Ubuqotho bocwaningo
Start · the discipline
Ubuqotho bocwaningo
- 04
Reproducibility
Start · the discipline
Reproducibility
- 05
Knowledge translation
Start · the discipline
Knowledge translation
- 01
-
02 Method and terms
- 06
From a prior to a…
Model · the learner
The probability that a published positive finding is true depends on the prior odds that the hypothesis was true, the study's power and…
- 07
Evidence synthesis
Record · the discipline
Bringing together the findings of many studies in a structured way.
- 09
Boundary organisation
Record · the discipline
An organisation that works between science and policy to keep evidence credible, relevant and legitimate.
- 11
Ubuqotho bocwaningo
Record · the discipline
Honesty, accountability and rigour in research.
- 06
-
04 Labs
- 08
Umfanekiso wedijithali
Exercise · the learner, on the faculty twin
The method runs on the faculty twin and the admitted map layers.
- 08
-
05 Assess
- 10
Module checks
Gate · the course
Each module ends with a short check against its outcomes.
- 12
Assessed work
Decision · the learner
The assessed work that closes a course: a lab record, a case memo or an assignment, reviewed against the rubric.
- 10
-
06 Record
- 13
Irekhodi lokuphothula
Outcome · the learner
The record stacks into the discipline achievement.
- 14
Discipline achievement
Record · the Academy
The discipline achievement follows the discipline rule, read from the records.
- 13
Research, evidence and the science-policy interface · Science, Technology and Society An outline of how the work is designed to run; it does not report the status of any decision or record.
07Umkhuba
Practice, learning and connections
Technology in this area
How technology helps and what risk it brings
| Aspect | In this area |
|---|---|
| Technology that helps | Foresight tools and regulatory sandboxes. |
| Risk technology creates | Threats to information integrity, and AI in public decisions. |
- RAS-421 Strategic Foresight and Horizon Scanning In development
- RAS-422 Responsible Innovation and Technology Assessment In development
- RAT-248 Information Integrity: Analysis and Response In development
- RAT-260 AI Assurance, Audit and Conformity Assessment In development
Impumelelo
Achievements in this discipline
Course achievements
One for each course with this as its primary discipline, earned when you pass the course's assessed activities.
Discipline: Research, evidence and the science-policy interface
Complete 3 assessed courses in research, evidence and the science-policy interface, at least 2 with it as their primary discipline and at least 1 at Intermediate (L2) or above, totalling at least 45 measured learning hours. In development.
Department: Science, Technology and Society
Earn two discipline achievements in the Department of Science, Technology and Society. In development.
Programme: Policy maker
Complete every core course of the Policy maker under assessed completion, ending with its capstone. In development.
An achievement is a record of the learning completed and assessed here. How recognition works.
Izindlela
Pathways that teach it
- Policy makerRole pathway2 of its 11 courses are in this discipline.
- Data and GeoAI specialistRole pathwayOne of its 12 courses is in this discipline.
- Nexus-Based Risk Management flagship pathwayAcademy flagshipOne of its 20 courses is in this discipline.
- Risk analystRole pathwayOne of its 11 courses is in this discipline.
Where this is practised
Where the subject is practised
- The Global Centre for Risk and Innovation (GCRI Canada)Research and evidence
- RegistryEvidence classes, Record provenance, Correction and versioning
- LabsReproducibility and negative results, Research protocols
- Purpose and publisherCountry and regional series, Editorial and corrections, Frontier technology series, Global systems series, Labs and foundry series, Scenarios and methods, Sectoral series, Universe annual synthesis
- The Global Risks Forum (GRF)Research NexusCollaborative research, Global research networks, Institutional knowledge gaps, Research integrity, Research methods dialogue, Risk knowledge translation, Risk research dialogue, The science-policy interface
Field platforms of the ecosystem where the subject is studied and applied.
Practised in
The two Foundry systems where it is practised
- FoundryNXS-LAB systemReproducible research
- NXS-HUB systemCitations, records and institutional memory
Foundry system pages describe what each system and its rooms are designed to do; the courses teach the methods these systems apply.
Across the ecosystem
Across the ecosystem
The science, platforms and standards this discipline draws on and contributes to across the ecosystem.
Foundations in the STEM Commons
- The Global Centre for Risk and Innovation (GCRI Canada)Policy, institutional and decision sciencePolicy, institutional and decision science studies how public decisions are made, implemented and evaluated.
- Causal inference and impact evaluationCausal inference and impact evaluation develop methods to determine whether and how interventions or exposures cause outcomes.
Where the field is worked on across the network
- The Global Centre for Risk and Innovation (GCRI Canada)RegistryThe platform's work on this field.
- LabsThe platform's work on this field.
Standards for the field
- Nexus StandardsData and EvidenceThe subsite that keeps the standards this discipline teaches.
- Readiness and AdoptionThe subsite that keeps the standards this discipline teaches.
Pages of the ecosystem, cited so learners can follow the field into research, platforms, standards and national practice.
08Bamba iqhaza
Take part in Governance, Policy and International Affairs
Qala kulokho ongikho.
Izindlela
Indlela
Izikhungo zikazwelonke
Iminyango kahulumeni, amahhovisi ezibalo nama-ejensi: idesiki likazwelonke nesahluko, nesihlalo emaqenjini okusebenza. Izahluko zikazwelonke
Indlela
Imikhakha nemisebenzi
Operators, regulators, banks, insurers and professional bodies whose work touches governance, policy and international affairs. For organisations
Indlela
Ukufunda
Izifundo nokuzijwayeza, nerekhodi lazo zonke izifundo eziqediwe. Mahhala ukufunda. Ukufunda okuvulekile
Indlela
Amazinga
Bhala okusalungiswa, buyekeza futhi wamukele izindlela namazinga edatha namaqembu okusebenza. Amaqembu okusebenza
Indlela
Imiphakathi nocwaningo
Universities, civil society and individual experts: projects, data and review. Write to the Academy
Izimemo ezivulekile
| Isimemo esivulekile | Kufunwa | Kuvalwa |
|---|---|---|
| Isimemo sokusungula sochwepheshe emaqenjini okusebenza CALL-FOUNDING-EXPERTS | uchwepheshe, umhleli, usihlalo | 2026-12-15 |
| Ukuqokwa okuvulekile kwezihlalo ezintathu zochwepheshe abazimele ze-Global Standards Council CALL-GSC-IND | ilungu lomkhandlu | 2026-12-01 |
| Ukuqokwa kwamalungu esifunda esikhashana (kusebenza kuphela uma isinqumo sesikhashana sithathwa) CALL-GSC-REG | ilungu lomkhandlu | 2026-12-15 |
| Ukubuyekezwa kwezinhlaka zomongo wengozi wegagasi 1 (C1, C2, C3, C4, C8, C10, C11) CALL-REVIEW-WAVE1 | umnikeli | 2027-01-31 |
| Ochwepheshe bamazinga ayisisekelo esayensi yengozi: ukungaqiniseki nokulinganiswa kobufakazi, izilinganiso, ukwahlulela kochwepheshe okuhlelekile, ukubekezeleleka, izimo ezeqisayo nezehlakalo ezihlanganisiwe, ingozi eyinhlekelele neyokuphila, izimiso nokulingana, amayunithi nokulinganisa (NRS-C17 kuya ku-NRS-C24) CALL-CORE-SCIENCE | uchwepheshe, umhleli, usihlalo | 2027-01-31 |
| Ababuyekezi abazimele bemindeni ye-atlas yezingozi, abathathu abavela okungenani ezifundeni ezimbili emndenini ngamunye (NHS2-R19), imindeni ehamba phambili kuqala CALL-ATLAS-REVIEW | uchwepheshe | 2027-03-31 |
| Amalungu e-Frontier Safety Review Panel yesikhashana nababuyekezi beqembu elibomvu (ukuphepha kwezinto eziphilayo nokuvikeleka kwazo, ukuphepha nokuvikeleka kwe-AI, ukuvikeleka kwengqalasizinda ebalulekile); izicelo ziphathwa ngokuyimfihlo CALL-FRONTIER-SAFETY-REVIEW | uchwepheshe | 2026-11-30 |
| Ochwepheshe be-ontology, i-metrology, ukuqinisekiswa kwesofthiwe nokuhlolwa kokuhambisana kwerejista yezidingo, ukumelwa komshini kwe-NHS-2, amasethi okuhlola ukuhambisana nohlelo lokuhlola ukuhambisana CALL-CONFORMANCE-ONTOLOGY | uchwepheshe, umhleli | 2027-01-31 |
Izimemo ezidlule usuku lwazo lokuvala zifihlwa ngokuzenzakalelayo. Zonke izimemo, iziphakamiso nezindlela: izindawo nezimemo ezivulekile.
Amaseli nama-CERT
Qoqa iseli
Hlanganisa abantu abathathu kuya kwabayisikhombisa abanamakhono ahambisanayo: idatha, ukusebenza enkundleni, inqubomgomo nokuxhumana.
Libhalise
Write to the team with the cell's name, members, area and region. You receive a reference and a contact. Write to the Academy
Khetha umqulu
Khetha ukugxila okukodwa: ucwaningo nobufakazi, idatha namamephu, ukubuyekezwa kwamazinga, noma ukulungiselela komphakathi.
Funda ndawonye
Landela izindlela zokufunda ezabelwana ngazo e-Risk Academy, nokuzijwayeza nerekhodi lazo zonke izifundo eziqediwe. Izinsongo nokuqina
Hola futhi ubike
Hold regular sessions. Publish what the cell learns through Reports. Bring findings to the working groups. Working groups
Gxila njengeThimba Lomphakathi Lokusabela Ezimweni Eziphuthumayo (CERT)
A CERT prepares its neighbourhood for the hazards it faces. It builds a local hazard profile from this site's map. It trains in household and street preparedness. It learns first aid and light search through recognised local providers, and it runs drills. A CERT works alongside the local emergency services and follows their direction. It never replaces them and is not an emergency service.
I-Index
What a national team or a local community can do on The Global Risks Index (NXS-GRX) today.
NXS-GRX
Your country's profile
Amaphrofayela engozi avulekile amazwe angu-195, nemithombo namalayisense kuyo yonke inani. Vula Inkomba
NXS-GRX
Lungisa noma uphendule
Lungisa inani noma uphendule ngezwe lakho. Ushintsho lurekhodwa kanye nomphumela walo. Vula Inkomba
NXS-GRX
Nikela ngedatha
Nikela ngeqoqo ledatha. Lingena ngesango lelayisense, ubongwa. Vula Inkomba
NXS-GRX
Sebenzisa idatha evulekile
Izendlalelo, amaphrofayela namathayela nge-API evulekile. Vula Inkomba
09References
Core references and where to begin
Start with Pielke, then Gluckman, then Ioannidis.
- Ioannidis, J. P. A., Why Most Published Research Findings Are False (2005). DOI
- Open Science Collaboration, Estimating the reproducibility of psychological science (2015). DOI
- Munafò, M. R. and others, A manifesto for reproducible science (2017). DOI
- Guyatt, G. H. and others, GRADE: an emerging consensus on rating quality of evidence and strength of recommendations (2008). DOI
- Page, M. J. and others, The PRISMA 2020 statement: an updated guideline for reporting systematic reviews (2021). DOI
- Wilkinson, M. D. and others, The FAIR Guiding Principles for scientific data management and stewardship (2016). DOI
- Pielke, R. A. Jr, The Honest Broker: Making Sense of Science in Policy and Politics (2007). DOI
- Gluckman, P., Policy: The art of science advice to government (2014). DOI
- Funtowicz, S. O. and Ravetz, J. R., Science for the post-normal age (1993). DOI
- Oliver, K., Innvar, S., Lorenc, T., Woodman, J. and Thomas, J., A systematic review of barriers to and facilitators of the use of evidence by policymakers (2014). DOI
10Imibuzo
Imibuzo, izimpendulo namagama abalulekile
Ukufunda lapha kungamahhala yini?
Yebo. Zonke izinkambo nezinto zokufunda ze-Risk Academy, eholwa yi-Nexus Consortium, zifundwa mahhala, ngesivinini sakho noma neqembu. Ukubamba iqhaza okujwayelekile akunayo inkokhelo; umsebenzi ohlawulelwayo, ukuqeqeshwa emsebenzini nezibonelelo zocwaningo kuyizinhlelo ezihlukile.
Kuyini engikuthola ngokuphothula isifundo?
Irekhodi lokuphothula: isifundo, inguqulo yaso, usuku nokuhlolwe, eliyimfihlo ngokuzenzakalelayo futhi labelwana nguwe. Lihlangana libe yimiphumela yomkhakha, yomnyango neyohlelo. Irekhodi likhomba okufundiwe nokuhlolwe, alisho ukuthi ukhono luqinisekisiwe; akulona ilayisense, isihloko sobuchwepheshe noma isimo somsebenzi.
Idatha nezindawo zivelaphi?
Ifakhalthi ngayinye ifundisa ngamawele endawo yangempela akhiwe ngedatha evulekile; isendlalelo ngasinye siveza umhlinzeki waso, ilayisense nosuku, futhi umphumela ngamunye ku-Academy uveza umthombo wawo. Amamephu nezibalo yizinkombandlela zokufundisa, ayisona isilinganiso sendawo.
How do I take part beyond the courses in research, evidence and the science-policy interface?
Yakha iseli lekhono labantu abathathu kuya kwabayisikhombisa, funda indlela okwabelwana ngayo, thatha umsebenzi ogadiwe kuma-National Working Groups, noma ulethe iqembu labafundi elivela esikhungweni sakho. Ezinhlanganweni.
Amagama abalulekile
- Evidence synthesis
- Bringing together the findings of many studies in a structured way.
- Boundary organisation
- An organisation that works between science and policy to keep evidence credible, relevant and legitimate.
- Ubuqotho bocwaningo
- Honesty, accountability and rigour in research.
- Reproducibility
- Obtaining the same results from the same data and methods.
ImibikoNikela
Nikela ngobufakazi kusuka kuleli khasi.
I-Reports iyincwadi yekota ye-Consortium. Lokho oakwaziyo ngalesi sihloko kungaba ubufakazi, icala noma ukulungisa kuhlelo olulandelayo, kubongwe wena futhi kushicilelwe ngaphansi kwe-CC BY 4.0 lapho kungumsebenzi wakho.
Risk Academy
- EmailGateway GW-ACADEMY Send it from this page Evidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen deposit Deposit it and submit to the community Publish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsRisk Academy See every way to contribute The contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Ungalokothi uthumele idatha yomuntu siqu, imininingwane ebucayi yengqalasizinda noma noma yini ongenalo ilungelo lokuyabelana ngayo. Umnikelo ubuyekezwa ngaphambi kokusetshenziswa; ukuwuthumela akunikezi ubuqambi noma amalungelo okuhlela, futhi ukusekela akuthengi muphumela.
Ucwaningo, ubufakazi nokuhlangana kwesayensi nenqubomgomo
Start with an open course
The catalogue lists each course with its outcomes, labs, hours and level.
Open RAF-103Back to Department of Science, Technology and Society