C5 · AI Security Engineering (core)
AI Security Evaluation and Authorized Assessment
Build reproducible evaluations, validate findings, and verify remediation on AI systems.
Proposed
0 of 6 modules accessible. Joining Skool does not unlock planned modules.
At a glance
- Audience
- Security practitioners taking responsibility for AI systems.
- Prerequisites
- C1–C4
- Ability to implement tests and interpret application behavior
- Planned duration
- 20 hours. Assumes the entry requirements are met. Planning estimate, not a measured completion time. Per module: 2 to 4 hours, plus practice.
- Assessment
- Scoped reproducible assessment with rubric.
- What is free
- The free foundation module listed on each course page, as it releases, plus community access.
Job tasks this course prepares you for
- Scope an AI-system assessment
- Design evaluations
- Validate impact
- Verify remediation
- Communicate uncertainty
Learning outcomes
- Produce an executable assessment plan with enforceable scope and stop conditions.
- Reproduce a seeded vulnerability with evidence of business impact.
- Build a repeatable evaluation suite with benign controls and held-out tests.
- Verify a fix without relying only on changes to the prompt that triggered the original failure.
Modules
00Authorization and test designProposed
Topics
- scope, accounts, data handling, stop conditions, business invariants.
01Attack-surface enumerationProposed
Topics
- application, retrieval, tool, memory, deployment, and identity interfaces.
02Adversarial casesProposed
Topics
- direct and indirect injection, entitlement bypass, tool misuse, data exposure, resource abuse.
03Evaluation methodsProposed
Topics
- repeated trials, baselines, held-out cases, scorer reliability, limits of judge models.
04Regression engineeringProposed
Topics
- model, prompt, tool, and configuration changes; CI integration; evidence retention.
05Reporting and retestProposed
Topics
- reproduction, impact, confidence, residual risk, remediation verification.
Labs and deliverables
- Assess a supplied application and agent stack. Test harness, findings, a remediation patch or specification, and retest evidence.
Labs run on SIFT, the forensics workstation Rob T. Lee created, on your own machine. Each module states what it needs before you start. Each module confirms its setup requirements and any cloud or API costs before it opens. About the SIFT lab environment.
Assessment
Scoped reproducible assessment with rubric. A named assessor grades against the published rubric (rubric to be published before release).
Pass criteria
- No scope violation
- Every seeded critical issue found or explicitly accounted for with evidence
- Reproducible execution
- 80% rubric
Critical failures
- Unauthorized cross-tenant access
- Out-of-scope activity
- Uncontrolled destructive action
- Fabricated evidence
- Exposed credentials
What this course does not teach
- No comprehensive penetration testing, offensive malware development, or full model-safety evaluation.