AI Red Teaming and Assurance

This course connects the work of probing an AI system with the case for putting it into service. It separates red teaming, the intentional effort to break, deceive, or abuse a system, from assurance, the documented evidence that a system merits trust for its intended use. Participants carry a full engagement from start to finish: matching test scope to the system's risk classification, setting objectives, writing rules of engagement so an exercise never turns into an incident, managing team makeup and deconfliction, and choosing techniques suited to each attack surface while weighing manual testing against automated tools.

Two features set the course apart. The first is agentic AI, where exposure extends beyond what a system says to the actions it takes. The second is assurance in its own right: participants build an auditable argument structured as claim, reasoning, and evidence, supported by TEV&V records and model and system cards, and follow how it feeds a risk-acceptance decision. Hard judgment calls are addressed head on, and scenario-driven exercises rather than a lab allow procurement and oversight staff to train side by side with testers.

What You Will Learn

• Distinguish red teaming from adjacent assurance activities, and place it correctly in the AI assurance lifecycle
• Run a red-team engagement lifecycle from planning through reporting
• Scope testing to a system's risk classification, and set objectives that produce usable results
• Write rules of engagement, and handle team composition and deconfliction
• Match adversarial testing techniques to the attack surface they actually address
• Weigh manual expert-driven testing against automated and tool-assisted approaches
• Apply probing techniques to generative AI systems
• Tell a genuine red-team finding apart from an ordinary software bug
• Build Test, Evaluation, Verification and Validation records that support a risk decision
• Construct an assurance case as claim, argument and evidence
• Interpret model cards and system cards, and judge independent evaluation results
• Structure a findings report and prioritize by impact rather than technical complexity
• Manage remediation and retesting, and apply human oversight as a mitigation
• Recognize when a finding becomes an incident
• Extend testing and assurance to foundation models and to agentic, tool-using systems

Course Outline

Module 1

• Foundations: What Is AI Red Teaming, and Where Does It Fit in AI Assurance?
• What red teaming is and is not
• Where it sits in the AI assurance lifecycle
• The range of assurance activities
• The guidance the course draws on
• Who is involved

Module 2

• Planning and Scoping a Red-Team Engagement
• The engagement lifecycle
• Scoping to risk classification
• Setting red-team objectives
• Rules of engagement
• Team composition and deconfliction

Module 3

• Adversarial Testing Techniques for Machine Learning and Generative AI
• Matching technique to attack surface
• Manual versus automated and tool-assisted red teaming
• Generative AI probing techniques
• A preview of testing agentic and tool-using systems
• Telling red-team findings apart from ordinary bugs

Module 4

• AI Assurance Frameworks: TEVV, Assurance Cases, and Documentation
• Test, Evaluation, Verification and Validation
• The assurance case as claim, argument and evidence
• Model cards and system cards
• Independent evaluation and measurement science
• From assurance documentation to a risk-acceptance decision

Module 5

• From Findings to Fixes: Reporting, Remediation, and Human Oversight
• Structuring a findings report
• Prioritizing by impact rather than technical complexity
• Remediation and retesting
• Human oversight as a mitigation category
• When a finding becomes an incident

Module 6

• Red Teaming and Assurance for Generative and Agentic AI; Capstone Scenario
• Foundation models and misuse-risk testing
• Agentic AI and the expansion of the attack surface from output to action
• Why red teaming and assurance are recurring rather than one-time
• A capstone scenario

This course is available for 365 days. You can choose when to start your access period. Once you activate, you will have 365 days to complete it (access the course materials, quizzes, projects and videos). You may request one extension of seven (7) days. Other extension requests will be evaluated on a case-by-case basis. Videos are not downloadable.