Risk Academy, led by the Nexus Consortium · Course catalogue · Technology and Digital Systems
مصنوعی ذہانت اور خودکار نظام
مصنوعی ذہانت کی حفاظت اور جائزہ
AI safety and evaluation, taught twice over: as the working methods of the field and as its frontier since 2024, with labs on the Academy's tools, open data and the Nexus Standards.
AI safety and evaluation: foundations and practice
This area covers how the capabilities, reliability and risks of AI systems are tested before and after deployment: evaluation design, red-teaming, benchmarks and their limits, safety cases for frontier models and agentic systems, and incident reporting.
AI safety and evaluation: foundations and practice
سطح
L2 Intermediate
Study time
about 24 hours (planning estimate)
حیثیت
In authoring
Prerequisites
An L1 course in the area, or an entry review
Why now
Shows why independent and held-out benchmarks matter when relying on vendor-reported evaluation results.
Define red-teaming and the other key terms of the discipline and use them correctly in a written case
Explain evaluation design and metrics and red-teaming and adversarial testing with a worked example and its source
Apply the discipline's method to a documented case and explain each result with its source
Analyse a named case against benchmarks and data contamination, stating evidence, date and limits
Modules
01
Evaluation design and metrics
Evaluation design and metrics
Red-teaming and adversarial testing
Benchmarks and data contamination
02
Dangerous capability evaluations
Dangerous capability evaluations
Safety cases for frontier models
Agentic system evaluation
03
Monitoring after deployment
Monitoring after deployment
Incident reporting and disclosure
04
Lab: the discipline's method
An attack success rate is measured per attempt against a stated attacker budget; the chance that at least one of k attempts succeeds grows as 1 minus (1 minus s) to the k, and a release gate compares that cumulative figure, after defences, with the level a safety case can tolerate.
Work the method through a documented case and explain each result
Explain what changed in AI safety and evaluation since 2024 and cite the studies, data or rules behind each change
Reproduce independent evaluation finds gap with developer self-reported scores on open data and state the uncertainty of the result
Evaluate a 2024 to 2026 study: its question, data, method, limits and what it changes for practice
Design an analysis that combines a frontier method with the faculty's instruments, and justify each choice
Modules
01
Independent evaluation finds gap with developer self-reported scores
NIST's Center for AI Standards and Innovation found DeepSeek V4 Pro scored better on the developer's own evaluations than on CAISI's, including non-public benchmarks, estimating it trails leading US models by about eight months.
Shows why independent and held-out benchmarks matter when relying on vendor-reported evaluation results.
National Institute of Standards and Technology, 1 May 2026
02
UK AI Security Institute publishes first Frontier AI Trends Report
Drawing on evaluations of over 30 frontier models since 2023, the institute reports fast capability growth in cyber, chemistry and biology tasks, universal jailbreaks found in every system tested, and rising self-replication precursors.
Offers government test results on dangerous capabilities and safeguard strength that evaluators can use to benchmark their own methods.
AI Security Institute (UK), 18 December 2025
03
Key update reviews technical safeguards and frontier risk management
The second key update finds prompt injection attacks still succeed about half the time with ten attempts, small data-poisoning attacks can be effective, and at least 12 companies now publish frontier safety frameworks with mostly implicit thresholds.
Clarifies how much assurance current safeguards provide, which shapes red-teaming scope and residual risk judgements.
International AI Safety Report, 25 November 2025
04
Time-horizon metric measures growth in autonomous task length
Researchers proposed measuring AI capability by the length of software tasks models complete reliably, and found this horizon has doubled roughly every seven months since 2019.
Gives evaluators a trend measure for agentic capability that can inform forecasts and thresholds in safety cases.
arXiv, 18 March 2025
05
UK renames AI Safety Institute as AI Security Institute
The UK government renamed the institute and refocused it on serious AI risks with security implications, such as chemical and biological misuse, cyber-attacks, fraud and child abuse material, adding a criminal misuse team.
Signals how a leading public evaluator now prioritises security-related misuse in its testing agenda.
UK Department for Science, Innovation and Technology, 14 February 2025
06
Reproduction lab
Extend the discipline's method with one frontier method and compare its results
State the uncertainty and the limits of the result
The Academy proposes how each discipline teaches toward the Nexus Standards; the mapping is open for public review. Bodies and standards are named so learners can find and apply their published work; the Academy teaches their use and links to the publisher, and no affiliation is implied.
04براہِ راست نقشہ
Connected activity in the United Kingdom and Ireland
Night-time lights and population density, the footprint of electrified, connected activity, over the United Kingdom and Ireland.
انٹرایکٹو نقشے کے لیے JavaScript درکار ہے؛ پرتوں کی فہرست اور ماخذات نیچے دیے گئے ہیں۔
Satellite image. Satellite or model raster as the provider publishes it. Clouds, swath gaps and retrieval error leave gaps and noise; values are indicators, not measurements at a place.
اس کے ذریعے رسائی
gibs.earthdata.nasa.gov، صرف اس وقت جب آپ براہ راست نقشے کی اجازت دیں
منتقل کرنے کے لیے گھسیٹیں۔ زوم کے لیے Ctrl یا ⌘ کے ساتھ اسکرول کریں، چٹکی لیں یا بٹن استعمال کریں؛ پوائنٹر کے نیچے اقدار کے لیے ہوور کریں؛ ری سیٹ خطے پر واپس لے جاتا ہے۔
Connected activity, framed on the United Kingdom and Ireland.Satellite and model products as NASA publishes them, each with its own date. They are indicators, not assessments of any place.
ہم NASA کی Global Imagery Browse Services (GIBS) کی خدمات سے فراہم کردہ تصاویر کے استعمال کا اعتراف کرتے ہیں، جو NASA کے Earth Science Data and Information System (ESDIS) کا حصہ ہے۔ رنگ شائع شدہ رنگی نقشے سے دوبارہ بنائے گئے؛ جہاں دکھائی گئی ہیں، اقدار کا H3 خانوں میں اوسط لیا گیا (ترمیم شدہ)۔ NASA کا کھلا ڈیٹا
ہم NASA کی Global Imagery Browse Services (GIBS) کی خدمات سے فراہم کردہ تصاویر کے استعمال کا اعتراف کرتے ہیں، جو NASA کے Earth Science Data and Information System (ESDIS) کا حصہ ہے۔ رنگ شائع شدہ رنگی نقشے سے دوبارہ بنائے گئے؛ جہاں دکھائی گئی ہیں، اقدار کا H3 خانوں میں اوسط لیا گیا (ترمیم شدہ)۔ NASA کا کھلا ڈیٹا
VIIRS Black Marble رات کی روشنیاں، NASA Goddard Space Flight Center؛ تقسیم کار NASA Level-1 and Atmosphere Archive and Distribution System (LAADS DAAC)؛ تصاویر بذریعہ NASA GIBS۔ رنگ شائع شدہ رنگی نقشے سے دوبارہ بنائے گئے؛ جہاں دکھائی گئی ہیں، اقدار کا H3 خانوں میں اوسط لیا گیا (ترمیم شدہ)۔ NASA کا کھلا ڈیٹا
Center for International Earth Science Information Network (CIESIN), Columbia University. 2018. Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11. NASA Socioeconomic Data and Applications Center (SEDAC). Licensed under CC BY 4.0. Colours redrawn from the published colour map; values averaged into H3 cells where shown (modified). doi:10.7927/H49C6VHW. CC BY 4.0
VIIRS کی تقریباً حقیقی وقت کی مصنوعات (سیلاب، زمینی سطح کا درجہ حرارت، برف کا احاطہ، درست شدہ انعکاس)، NASA LANCE؛ تصاویر بذریعہ NASA GIBS۔ رنگ شائع شدہ رنگی نقشے سے دوبارہ بنائے گئے؛ جہاں دکھائی گئی ہیں، اقدار کا H3 خانوں میں اوسط لیا گیا (ترمیم شدہ)۔ NASA کا کھلا ڈیٹا
ASTER GDEM، METI اور NASA کی پروڈکٹ ہے (ASTER Global Digital Elevation Model V003، NASA LP DAAC)؛ تصاویر بذریعہ NASA GIBS۔ رنگ شائع شدہ رنگی نقشے سے دوبارہ بنائے گئے؛ جہاں دکھائی گئی ہیں، اقدار کا H3 خانوں میں اوسط لیا گیا (ترمیم شدہ)۔ حوالہ: NASA/METI/AIST/Japan Spacesystems and U.S./Japan ASTER Science Team (2019)۔ ASTER Global Digital Elevation Model V003۔ doi:10.5067/ASTER/ASTGTM.003. NASA کا کھلا ڈیٹا
Natural Earth کے ساتھ تیار کردہ۔ مفت ویکٹر اور راسٹر نقشہ ڈیٹا @ naturalearthdata.com۔ عوامی ملکیت
Blue Marble، NASA Earth Observatory؛ تصاویر بذریعہ NASA GIBS۔ رنگ شائع شدہ رنگی نقشے سے دوبارہ بنائے گئے؛ جہاں دکھائی گئی ہیں، اقدار کا H3 خانوں میں اوسط لیا گیا (ترمیم شدہ)۔ NASA کا کھلا ڈیٹا
براہ راست واقعات: NASA Earth Observatory Natural Event Tracker (EONET)؛ واقعات اشاریاتی ہیں اور اپنے اصل ماخذ کے لنکس برقرار رکھتے ہیں۔
NASA GIBS: ہم NASA کی Global Imagery Browse Services (GIBS) کی خدمات سے فراہم کردہ تصاویر کے استعمال کا اعتراف کرتے ہیں، جو NASA کے Earth Science Data and Information System (ESDIS) کا حصہ ہے۔
اختیاری پرتیں (پرتوں کی فہرست میں 3 ماخذات: براہ راست، پیش گوئی، مشاہدہ شدہ اور سیاق) فعال ہونے پر اپنے فراہم کنندہ کا حوالہ نقشے پر اور یہاں درج کرتی ہیں؛ پیش گوئیاں ماڈل کا نتیجہ ہیں اور اخذ شدہ اشاریے تخمینے ہیں۔
پرتیں اور تاریخیں
رات کی روشنیاں، یومیہ: VIIRS Black Marble (VNP46A2)، NASA LAADS DAAC · 12 اکتوبر 2026
Population density, 2020: Population: CIESIN, Columbia University, Gridded Population of the World v4.11 (2018), NASA SEDAC, CC BY 4.0 · static
تعلیم کیسے چلتی ہے، ایک ایسے عمل کے طور پر جسے آپ چلا سکتے ہیں
8 steps · 4 stages · 2 gates · 16 processes
ایک منظرنامہ منتخب کریں اور چلائیں دبائیں: کورس، راستہ، مشق، فیکلٹی، سیلز اور ریکارڈ۔
Risk Academy
Course, catalogue to completion
A free course pairs one discipline's methods with a real place, open data and standards, with module checks, assessed work and a completion record.
Scenarios Course · Practice · Record
01 Open
01
Choose a course
Start · the learner
Find the course in the catalogue, by faculty, level, kind or the standard it teaches.
02
Entry check
Gate · the course
The prerequisites say what to study first; an entry review stands in for them.
02 Study
03
اسباق
Step · the learner
Read and work through the lessons, self-paced or with a cohort.
04
Labs on the twin
Exercise · the learner, on the faculty twin
Run the discipline's method and the faculty instruments on the twin of a real place.
03 Assess
05
Module checks
Gate · the course
Each module ends with a short check against its outcomes.
06
Assessed work
Decision · reviewers and peers
A lab record, a case memo and a key terms check, with peer review where the course asks for it.
04 Record
07
تکمیل کا ریکارڈ
Record · the Academy
The record names the course, its version and the date; it is the learner's to share.
08
Next course
Outcome · the learner
The record stacks into the discipline and pathway achievements, and the next course opens.
How learning works · Courses An outline of how the work is designed to run; it does not report the status of any decision or record.
06حصہ لیں
کام میں حصہ لیں
اپنی شناخت سے شروع کریں۔
Routes
راستہ
قومی ادارے
Ministries, statistics offices and agencies: a national desk and chapter, and a seat in the working groups. National chapters
راستہ
Sectors and professions
Operators, regulators, banks, insurers and professional bodies whose work touches systemic risk. For organisations
راستہ
تعلیم
Courses and practice, with a record of every course completed. Free to study. Open learning
راستہ
معیارات
Draft, review and adopt the methods and data standards with the working groups. Working groups
راستہ
Communities and research
Universities, civil society and individual experts: projects, data and review. Write to the Academy
Open calls
Open call
Seeks
Closes
Founding call for experts to the working groups CALL-FOUNDING-EXPERTS
expert, editor, chair
2026-12-15
Open nomination for the three independent expert seats of the Global Standards Council CALL-GSC-IND
کونسل کا رکن
2026-12-01
Nominations of interim regional members (applies only if the transitional provision is decided) CALL-GSC-REG
کونسل کا رکن
2026-12-15
Review of the wave-1 risk core outlines (C1, C2, C3, C4, C8, C10, C11) CALL-REVIEW-WAVE1
contributor
2027-01-31
Experts for the core standards of risk science: uncertainty and evidence grading, scales, structured expert judgement, tolerability, extremes and compound events, catastrophic and existential risk, ethics and equity, units and measurement (NRS-C17 to NRS-C24) CALL-CORE-SCIENCE
expert, editor, chair
2027-01-31
Independent reviewers for the families of the hazard atlas, three from at least two regions for each family (NHS2-R19), priority families first CALL-ATLAS-REVIEW
ماہر
2027-03-31
Members of the interim Frontier Safety Review Panel and red-team reviewers (biosafety and biosecurity, AI safety and security, critical-infrastructure security); applications treated in confidence CALL-FRONTIER-SAFETY-REVIEW
ماہر
2026-11-30
Experts in ontology, metrology, software verification and conformity assessment for the requirement register, the NHS-2 machine representation, the conformance test suites and the conformity assessment scheme CALL-CONFORMANCE-ONTOLOGY
expert, editor
2027-01-31
Calls past their closing date are hidden automatically. Every call, proposal and route: open areas and calls.
Cells
A cell is a small team with a clear mission, a host and a record of what it does. Choose the type that fits.
University, student and national chapter teams. Open
A cell acts within its host's rules and never replaces an authority. A CERT works alongside the local emergency services, follows their direction and is not an emergency service.
How a cell forms
01
Gather the cell
Bring together three to seven people with complementary skills: data, field practice, policy and communication.
02
Choose a type
Pick the kind of cell that fits the work, from the gallery below. A cell can add a second focus later. See the types
03
Register it
Name the cell, its members, area and region, and who hosts it. You receive a reference and a contact. Form a cell
04
Learn together
Follow shared study paths at Risk Academy, with practice and a record of every course completed. Hazards and resilience
05
Lead and report
Hold regular sessions. Publish what the cell learns through Reports. Bring findings to the working groups. Working groups
انڈیکس
What a national team or a local community can do on The Global Risks Index (NXS-GRX) today.
NXS-GRX
Your country's profile
Open risk profiles for 195 countries, with sources and licences on every value. Open the Index
NXS-GRX
Correct or reply
Correct a value or reply for your country. The change is logged with its effect. Open the Index
NXS-GRX
Offer data
Offer a dataset. It enters through the licence gate, with your credit. Open the Index
NXS-GRX
Use the open data
Layers, profiles and tiles through the open API. Open the Index
07سوالات
سوالات، جوابات اور کلیدی اصطلاحات
Is the course free?
جی ہاں۔ Risk Academy، جس کی قیادت Nexus Consortium کرتا ہے، کا ہر کورس اور تعلیمی مواد مفت ہے، اپنی رفتار سے یا کسی گروپ کے ساتھ۔ عام شرکت کی کوئی فیس نہیں؛ معاوضے والا کام، انٹرن شپس اور فیلوشپس الگ انتظامات ہیں۔
کورس مکمل کرنے سے مجھے کیا ملتا ہے؟
A completion record: the course, its version, the date and what was assessed, private by default and shared by you. It stacks into discipline, department and programme achievements. A record states what was studied and assessed and does not certify competence; it is not a licence, a professional title or an employment status.
ڈیٹا اور مقامات کہاں سے آتے ہیں؟
Every faculty teaches on a twin of a real place built from open data; each layer carries its provider, licence and date, and every result on the Academy states its source. Maps and figures are indicators for teaching, never an assessment of a place.
How do I take part beyond the course?
Form a competence cell of three to seven people, study a shared pathway, take a supervised assignment in the National Working Groups, or bring a cohort from your institution. For organisations.
کلیدی اصطلاحات
Red-teaming
Structured attempts to make a system fail or misbehave, to find weaknesses before others do.
Benchmark contamination
When test data have leaked into a model's training data, so scores overstate its ability.
Safety case
A structured argument, supported by evidence, that a system is acceptably safe for a given use.
Frontier model
A general-purpose AI model at or near the most capable level available at a given time.
رپورٹیںتعاون کریں
اس صفحے سے شواہد فراہم کریں۔
Reports is the Consortium's quarterly publication, written by six desks. This page sits with the Academy Capability desk. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work. Sent to the wrong desk, it is forwarded, not refused.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
یہ سائٹ صرف لازمی طور پر ضروری کوکیز استعمال کرتی ہے تاکہ کام کر سکے اور آپ کے رازداری کے انتخاب یاد رکھ سکے۔ دوسری کمپنیوں سے کچھ بھی لوڈ نہیں ہوتا، اور ہم آپ کی ذاتی معلومات کبھی فروخت یا شیئر نہیں کرتے۔
تفصیلات
یہ سائٹ صرف لازمی طور پر ضروری کوکیز استعمال کرتی ہے تاکہ کام کر سکے اور آپ کے رازداری کے انتخاب یاد رکھ سکے۔ دوسری کمپنیوں سے کچھ بھی لوڈ نہیں ہوتا، اور ہم آپ کی ذاتی معلومات کبھی فروخت یا شیئر نہیں کرتے۔ ہم کوئی اشتہار نہیں چلاتے۔ آپ ہر صفحے کے آخر میں "رازداری کے انتخاب" کے ذریعے کسی بھی وقت اپنے انتخاب کا جائزہ لے سکتے ہیں۔