Risk Academy, led by the Nexus Consortium · Course catalogue · Technology and Digital Systems
Kunsmatige intelligensie en outonome stelsels
KI-veiligheid en evaluering
AI safety and evaluation, taught twice over: as the working methods of the field and as its frontier since 2024, with labs on the Academy's tools, open data and the Nexus Standards.
AI safety and evaluation: foundations and practice
This area covers how the capabilities, reliability and risks of AI systems are tested before and after deployment: evaluation design, red-teaming, benchmarks and their limits, safety cases for frontier models and agentic systems, and incident reporting.
AI safety and evaluation: foundations and practice
Level
L2 Intermediate
Study time
about 24 hours (planning estimate)
Status
In authoring
Prerequisites
An L1 course in the area, or an entry review
Why now
Shows why independent and held-out benchmarks matter when relying on vendor-reported evaluation results.
Define red-teaming and the other key terms of the discipline and use them correctly in a written case
Explain evaluation design and metrics and red-teaming and adversarial testing with a worked example and its source
Apply the discipline's method to a documented case and explain each result with its source
Analyse a named case against benchmarks and data contamination, stating evidence, date and limits
Modules
01
Evaluation design and metrics
Evaluation design and metrics
Red-teaming and adversarial testing
Benchmarks and data contamination
02
Dangerous capability evaluations
Dangerous capability evaluations
Safety cases for frontier models
Agentic system evaluation
03
Monitoring after deployment
Monitoring after deployment
Incident reporting and disclosure
04
Lab: the discipline's method
An attack success rate is measured per attempt against a stated attacker budget; the chance that at least one of k attempts succeeds grows as 1 minus (1 minus s) to the k, and a release gate compares that cumulative figure, after defences, with the level a safety case can tolerate.
Work the method through a documented case and explain each result
Explain what changed in AI safety and evaluation since 2024 and cite the studies, data or rules behind each change
Reproduce independent evaluation finds gap with developer self-reported scores on open data and state the uncertainty of the result
Evaluate a 2024 to 2026 study: its question, data, method, limits and what it changes for practice
Design an analysis that combines a frontier method with the faculty's instruments, and justify each choice
Modules
01
Independent evaluation finds gap with developer self-reported scores
NIST's Center for AI Standards and Innovation found DeepSeek V4 Pro scored better on the developer's own evaluations than on CAISI's, including non-public benchmarks, estimating it trails leading US models by about eight months.
Shows why independent and held-out benchmarks matter when relying on vendor-reported evaluation results.
National Institute of Standards and Technology, 1 May 2026
02
UK AI Security Institute publishes first Frontier AI Trends Report
Drawing on evaluations of over 30 frontier models since 2023, the institute reports fast capability growth in cyber, chemistry and biology tasks, universal jailbreaks found in every system tested, and rising self-replication precursors.
Offers government test results on dangerous capabilities and safeguard strength that evaluators can use to benchmark their own methods.
AI Security Institute (UK), 18 December 2025
03
Key update reviews technical safeguards and frontier risk management
The second key update finds prompt injection attacks still succeed about half the time with ten attempts, small data-poisoning attacks can be effective, and at least 12 companies now publish frontier safety frameworks with mostly implicit thresholds.
Clarifies how much assurance current safeguards provide, which shapes red-teaming scope and residual risk judgements.
International AI Safety Report, 25 November 2025
04
Time-horizon metric measures growth in autonomous task length
Researchers proposed measuring AI capability by the length of software tasks models complete reliably, and found this horizon has doubled roughly every seven months since 2019.
Gives evaluators a trend measure for agentic capability that can inform forecasts and thresholds in safety cases.
arXiv, 18 March 2025
05
UK renames AI Safety Institute as AI Security Institute
The UK government renamed the institute and refocused it on serious AI risks with security implications, such as chemical and biological misuse, cyber-attacks, fraud and child abuse material, adding a criminal misuse team.
Signals how a leading public evaluator now prioritises security-related misuse in its testing agenda.
UK Department for Science, Innovation and Technology, 14 February 2025
06
Reproduction lab
Extend the discipline's method with one frontier method and compare its results
State the uncertainty and the limits of the result
The Academy proposes how each discipline teaches toward the Nexus Standards; the mapping is open for public review. Bodies and standards are named so learners can find and apply their published work; the Academy teaches their use and links to the publisher, and no affiliation is implied.
04Lewendige kaart
Connected activity in the United Kingdom and Ireland
Night-time lights and population density, the footprint of electrified, connected activity, over the United Kingdom and Ireland.
Die interaktiewe kaart benodig JavaScript; die laaglys en bronne is hieronder.
daagliks; Satellietstraling, gekorrigeer vir maanlig en atmosfeer
Nuutste by bou
2026-10-12
Resolusie
ongeveer 500 m-rooster
Onsekerheid
Satelliet-nagstraling vir die getoonde datum; ligte is ’n benadering van elektrisiteitsgebruik en onderbrekings, nie ’n meting van voorsiening of toegang nie.
Bereik deur
gibs.earthdata.nasa.gov, only after you allow the live map
Oor hierdie laag
Bewyse
Geskat
Bron
Gridded Population of the World v4.11 (NASA SEDAC (CIESIN, Columbia-universiteit))
Sensusgebaseerde skatting van mense per vierkante kilometer in 2020, getoon as blootstellingskonteks; nie ’n telling van mense wat deur enige gevaar geraak word nie.
Bereik deur
gibs.earthdata.nasa.gov, only after you allow the live map
Satellietbeeld. Satelliet- of modelraster soos die verskaffer dit publiseer. Wolke, strookgapings en herwinningsfoute laat gapings en geraas; waardes is aanwysers, nie metings op 'n plek nie.
Bereik deur
gibs.earthdata.nasa.gov, only after you allow the live map
Sleep om te skuif. Ctrl of ⌘ en rol, knyp of die knoppies om te zoem; sweef vir die waardes onder die wyser; terugstel keer terug na die streek.
Connected activity, framed on the United Kingdom and Ireland.Satellite and model products as NASA publishes them, each with its own date. They are indicators, not assessments of any place.
Ons erken die gebruik van beelde wat deur dienste van NASA se Global Imagery Browse Services (GIBS) verskaf word, deel van NASA se Earth Science Data and Information System (ESDIS). Kleure herteken vanaf die gepubliseerde kleurkaart; waardes waar aangedui in H3-selle gemiddeld (gewysig). NASA-oopdata
Ons erken die gebruik van beelde wat deur dienste van NASA se Global Imagery Browse Services (GIBS) verskaf word, deel van NASA se Earth Science Data and Information System (ESDIS). Kleure herteken vanaf die gepubliseerde kleurkaart; waardes waar aangedui in H3-selle gemiddeld (gewysig). NASA-oopdata
VIIRS Black Marble-naglig, NASA Goddard Space Flight Center; versprei deur die NASA Level-1 and Atmosphere Archive and Distribution System (LAADS DAAC); beelde via NASA GIBS. Kleure herteken vanaf die gepubliseerde kleurkaart; waardes waar aangedui in H3-selle gemiddeld (gewysig). NASA-oopdata
Center for International Earth Science Information Network (CIESIN), Columbia University. 2018. Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11. NASA Socioeconomic Data and Applications Center (SEDAC). Licensed under CC BY 4.0. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). doi:10.7927/H49C6VHW. CC BY 4.0
VIIRS byna intydse produkte (vloed, landoppervlaktemperatuur, sneeudekking, gekorrigeerde weerkaatsing), NASA LANCE; beelde via NASA GIBS. Kleure herteken vanaf die gepubliseerde kleurkaart; waardes waar aangedui in H3-selle gemiddeld (gewysig). NASA-oopdata
ASTER GDEM is 'n produk van METI en NASA (ASTER Global Digital Elevation Model V003, NASA LP DAAC); beelde via NASA GIBS. Kleure herteken vanaf die gepubliseerde kleurkaart; waardes waar aangedui in H3-selle gemiddeld (gewysig). Haal aan as: NASA/METI/AIST/Japan Spacesystems and U.S./Japan ASTER Science Team (2019). ASTER Global Digital Elevation Model V003. doi:10.5067/ASTER/ASTGTM.003. NASA-oopdata
Gemaak met Natural Earth. Gratis vektor- en rasterkaartdata @ naturalearthdata.com. Openbare domein
Blue Marble, NASA Earth Observatory; beelde via NASA GIBS. Kleure herteken vanaf die gepubliseerde kleurkaart; waardes waar aangedui in H3-selle gemiddeld (gewysig). NASA-oopdata
Lewende gebeure: NASA Earth Observatory Natural Event Tracker (EONET); gebeure is aanduidend en behou hul oorspronklike bronskakels.
NASA GIBS: Ons erken die gebruik van beelde wat deur dienste van NASA se Global Imagery Browse Services (GIBS) verskaf word, deel van NASA se Earth Science Data and Information System (ESDIS).
Opsionele lae (3 bronne in die laaglys: lewend, voorspelling, waargeneem en konteks) druk hul verskaffererkenning op die kaart en hier af wanneer hulle aangeskakel word; voorspellings is modeluitsette en afgeleide indekse is skattings.
Lae en datums
Naglig, daagliks: VIIRS Black Marble (VNP46A2), NASA LAADS DAAC · 12 Okt. 2026
Population density, 2020: Population: CIESIN, Columbia University, Gridded Population of the World v4.11 (2018), NASA SEDAC, CC BY 4.0 · static
Ware kleur: VIIRS NOAA-20 gekorrigeerde weerkaatsing, NASA LANCE · 13 Oktober 2026
Kies 'n scenario en druk Speel: kursus, leerpad, praktyk, fakulteit, selle en rekord.
Risk Academy
Course, catalogue to completion
A free course pairs one discipline's methods with a real place, open data and standards, with module checks, assessed work and a completion record.
Scenarios Course · Practice · Record
01 Open
01
Choose a course
Start · the learner
Find the course in the catalogue, by faculty, level, kind or the standard it teaches.
02
Entry check
Gate · the course
The prerequisites say what to study first; an entry review stands in for them.
02 Study
03
Lesse
Step · the learner
Read and work through the lessons, self-paced or with a cohort.
04
Labs on the twin
Exercise · the learner, on the faculty twin
Run the discipline's method and the faculty instruments on the twin of a real place.
03 Assess
05
Module checks
Gate · the course
Each module ends with a short check against its outcomes.
06
Assessed work
Decision · reviewers and peers
A lab record, a case memo and a key terms check, with peer review where the course asks for it.
04 Record
07
Voltooiingsrekord
Record · the Academy
The record names the course, its version and the date; it is the learner's to share.
08
Next course
Outcome · the learner
The record stacks into the discipline and pathway achievements, and the next course opens.
How learning works · Courses An outline of how the work is designed to run; it does not report the status of any decision or record.
06Neem deel
Neem aan die werk deel
Begin by wie u is.
Roetes
Roete
Nasionale instellings
Ministeries, statistiekkantore en agentskappe: 'n nasionale lessenaar en tak, en 'n setel in die werkgroepe. Nasionale hoofstukke
Roete
Sektore en beroepe
Operateurs, reguleerders, banke, versekeraars en professionele liggame wie se werk sistemiese risiko raak. Vir organisasies
Roete
Leer
Kursusse en praktyk, met 'n rekord van elke voltooide kursus. Gratis om te studeer. Oop leer
Roete
Standaarde
Stel die metodes en datastandaarde saam met die werkgroepe op, hersien en aanvaar dit. Werkgroepe
Roete
Gemeenskappe en navorsing
Universities, civil society and individual experts: projects, data and review. Write to the Academy
Oop oproepe
Oop oproep
Soek
Sluit
Stigtingsoproep vir kundiges vir die werkgroepe CALL-FOUNDING-EXPERTS
kundige, redakteur, voorsitter
2026-12-15
Oop nominasie vir die drie onafhanklike kundigesetels van die Wêreldwye Standaarderaad CALL-GSC-IND
raadslid
2026-12-01
Nominasies van tussentydse streekslede (geld slegs as daar oor die oorgangsbepaling besluit word) CALL-GSC-REG
raadslid
2026-12-15
Hersiening van die uiteensettings van die golf-1-risikokern (C1, C2, C3, C4, C8, C10, C11) CALL-REVIEW-WAVE1
bydraer
2027-01-31
Kundiges vir die kernstandaarde van risikowetenskap: onsekerheid en gradering van bewyse, skale, gestruktureerde kundige oordeel, aanvaarbaarheid, uiterstes en saamgestelde gebeure, katastrofiese en eksistensiële risiko, etiek en billikheid, eenhede en meting (NRS-C17 tot NRS-C24) CALL-CORE-SCIENCE
kundige, redakteur, voorsitter
2027-01-31
Onafhanklike beoordelaars vir die families van die gevaar-atlas, drie uit minstens twee streke vir elke familie (NHS2-R19), prioriteitsfamilies eerste CALL-ATLAS-REVIEW
kundige
2027-03-31
Lede van die tussentydse Frontier Safety Review Panel en rooispanbeoordelaars (biologiese veiligheid en biosekuriteit, KI-veiligheid en -sekuriteit, sekuriteit van kritieke infrastruktuur); aansoeke word vertroulik hanteer CALL-FRONTIER-SAFETY-REVIEW
kundige
2026-11-30
Kundiges in ontologie, metrologie, sagtewareverifikasie en konformiteitsbeoordeling vir die vereisteregister, die NHS-2-masjienvoorstelling, die konformiteitstoetsreekse en die konformiteitsbeoordelingskema CALL-CONFORMANCE-ONTOLOGY
kundige, redakteur
2027-01-31
Oproepe ná hul sluitingsdatum word outomaties versteek. Elke oproep, voorstel en roete: oop gebiede en oproepe.
Cells
A cell is a small team with a clear mission, a host and a record of what it does. Choose the type that fits.
University, student and national chapter teams. Open
A cell acts within its host's rules and never replaces an authority. A CERT works alongside the local emergency services, follows their direction and is not an emergency service.
How a cell forms
01
Bring die sel bymekaar
Bring drie tot sewe mense met aanvullende vaardighede bymekaar: data, veldpraktyk, beleid en kommunikasie.
02
Choose a type
Pick the kind of cell that fits the work, from the gallery below. A cell can add a second focus later. See the types
03
Registreer dit
Name the cell, its members, area and region, and who hosts it. You receive a reference and a contact. Form a cell
04
Leer saam
Volg gedeelde studiepaaie by Risk Academy, met praktyk en 'n rekord van elke voltooide kursus. Gevare en veerkragtigheid
05
Lei en rapporteer
Hold regular sessions. Publish what the cell learns through Reports. Bring findings to the working groups. Working groups
Die Indeks
What a national team or a local community can do on The Global Risks Index (NXS-GRX) today.
NXS-GRX
Your country's profile
Oop risikoprofiele vir 195 lande, met bronne en lisensies by elke waarde. Open die Indeks
NXS-GRX
Korrigeer of antwoord
Korrigeer 'n waarde of antwoord namens jou land. Die verandering word met sy effek aangeteken. Open die Indeks
NXS-GRX
Bied data aan
Bied 'n datastel aan. Dit kom deur die lisensiehek in, met erkenning aan jou. Open die Indeks
Ja. Elke kursus en leermateriaal van Risk Academy, gelei deur die Nexus Consortium, is gratis om te studeer, op eie tempo of saam met 'n groep. Gewone deelname het geen fooi nie; betaalde werk, internskappe en genootskappe is afsonderlike reëlings.
Wat gee die voltooiing van 'n kursus my?
'n Voltooiingsrekord: die kursus, sy weergawe, die datum en wat geassesseer is, by verstek privaat en deur u gedeel. Dit stapel op tot dissipline-, departements- en programprestasies. 'n Rekord meld wat bestudeer en geassesseer is en sertifiseer nie bevoegdheid nie; dit is nie 'n lisensie, 'n professionele titel of 'n indiensnemingstatus nie.
Waar kom die data en die plekke vandaan?
Elke fakulteit onderrig op 'n tweeling van 'n werklike plek wat uit oop data gebou is; elke laag dra sy verskaffer, lisensie en datum, en elke resultaat op die Akademie vermeld sy bron. Kaarte en figure is aanwysers vir onderrig, nooit 'n assessering van 'n plek nie.
How do I take part beyond the course?
Vorm 'n bevoegdheidsel van drie tot sewe mense, bestudeer 'n gedeelde roete, neem 'n opdrag onder toesig in die Nasionale Werkgroepe, of bring 'n groep van jou instelling. Vir organisasies.
Kernterme
Red-teaming
Structured attempts to make a system fail or misbehave, to find weaknesses before others do.
Benchmark contamination
When test data have leaked into a model's training data, so scores overstate its ability.
Safety case
A structured argument, supported by evidence, that a system is acceptably safe for a given use.
Frontier model
A general-purpose AI model at or near the most capable level available at a given time.
VerslaeDra by
Dra bewyse vanaf hierdie bladsy by.
Reports is the Consortium's quarterly publication, written by six desks. This page sits with the Academy Capability desk. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work. Sent to the wrong desk, it is forwarded, not refused.
Stuur nooit persoonlike data, sensitiewe infrastruktuurbesonderhede of enigiets wat u geen reg het om te deel nie. 'n Bydrae word voor gebruik beoordeel; die stuur daarvan verleen geen outeurskap of redaksionele regte nie, en borgskap koop nooit 'n bevinding nie.
Hierdie werf gebruik slegs streng noodsaaklike koekies om te werk en om jou privaatheidskeuses te onthou. Niks word van ander maatskappye gelaai nie, en ons verkoop of deel nooit jou persoonlike inligting nie.
Besonderhede
Hierdie werf gebruik slegs streng noodsaaklike koekies om te werk en om jou privaatheidskeuses te onthou. Niks word van ander maatskappye gelaai nie, en ons verkoop of deel nooit jou persoonlike inligting nie. Ons voer geen advertensies nie. Jy kan jou keuses enige tyd hersien deur Privaatheidskeuses onderaan elke bladsy.