AI assurance and audit: foundations and practice
- Level
- L2 Intermediate
- Study time
- about 24 hours (planning estimate)
- অবস্থা
- Planned
- Prerequisites
- An L1 course in the area, or an entry review
Changes compliance timelines and planning assumptions for organisations deploying high-risk AI systems in the EU.
Learning outcomes
- Define AI assurance and the other key terms of the discipline and use them correctly in a written case
- Explain AI system impact assessment and AI risk management frameworks with a worked example and its source
- Apply the discipline's method to a documented case and explain each result with its source
- Analyse a named case against auditing AI management systems, stating evidence, date and limits
Modules
-
AI system impact assessment
- AI system impact assessment
- AI risk management frameworks
- Auditing AI management systems
-
Conformity assessment under the EU AI Act
- Conformity assessment under the EU AI Act
- Assurance cases and evidence
- Documentation: model cards, system cards and technical files
-
Agentic AI assurance and oversight
- Agentic AI assurance and oversight
- Red teaming and third-party evaluation
- Post-market monitoring and incident reporting
-
Lab: the discipline's method
- Control coverage is two figures, designed and operating; operating effectiveness is judged from a sample of records, and the sample size bounds the exception rate an auditor can rule out, so the opinion rests on how many records were pulled as much as on what they showed.
- Work the method through a documented case and explain each result
- Read the method's sources and stated values
Labs with Academy tools
পদ্ধতি
- From a control list to an audit opinion: what a sample of records can establishThe discipline's method on the discipline page, with its cited sources, worked through a documented case
উপকরণ
- Orbit propagatorA faculty instrument on the faculty page, run on the faculty's digital twin data
- Evaluation ledgerA faculty instrument on the faculty page, run on the faculty's digital twin data
Twin builder
- Digital twin builderBuild the faculty's place in 3D, switch layers and read the credits and dates of each
Map layer
- Night lights tonight, outages (VIIRS Day/Night Band)An open map layer the Academy's licence register admits, read in the map tools
- Built-up and settlement extent, 2010 (HBASE, Landsat)An open map layer the Academy's licence register admits, read in the map tools
- Population density, 2020 (GPW v4.11)An open map layer the Academy's licence register admits, read in the map tools
Foundry room
- Assurance room, NXS-NSFAssurance cases and audit findings for AI systems
- Benchmarks room, NXS-LABThird-party evaluation of AI systems on stated tasks
Assessment
- 40%
Lab record
Work the discipline's method through a documented case and explain each result with its source and date
- 40%
Case memo
Apply the discipline's methods to a named place or organisation, stating evidence and limits
- 20%
Key terms check
Short answers on the key terms, marked against the definitions
Delivery
- Self-paced online, with the labs in the browser
- Cohort study over six to eight weeks, with a weekly lab review
- Co-delivered with a partner institution, inside its own courses
Partner on this course
- Co-teach a module and bring a current case from your work
- Host a cohort for your learners, with your own schedule and review
- Contribute a lab: a dataset your institution publishes openly, or a model
- Review the course content and its references before it opens
- Translate the course and adapt its cases to your region
How institutions and experts take part
In the Academy LMS
- Course on the learning platform
- Intermediate, 4 sections
- Cohort group
- A community group for each cohort of learners
- Achievement
- Recorded on completion of the assessed work
References 4
- Council of the European Union, Artificial intelligence: Council and Parliament agree to simplify and streamline rules,
- Raji et al., Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing, 2020
- Mökander, Schuett, Kirk and Floridi, Auditing large language models: a three-layered approach, 2023
- Schuett, Three lines of defense against risks from AI, 2023