aiops models
Inspect the simulated model catalog, manifests, and version comparisons.
AI operations lab
Evaluations and datasets
Operate golden sets, evaluation runs, thresholds, regressions, slices, annotations, and release evidence.
Track progress
0/180 lessons
0/18 missions
0/11 briefings
12 courses
01
Evaluations and datasets
Inspect a golden dataset
Inspect a golden dataset through a bounded evidence-first AI operations workflow in the training lab.
Core 02 Evaluations and datasets Read evaluation casesRead evaluation cases through a bounded evidence-first AI operations workflow in the training lab.
Core Checkpoint 03 Evaluations and datasets Validate a dataset schemaValidate a dataset schema through a bounded evidence-first AI operations workflow in the training lab.
Core 04 Evaluations and datasets Count evaluation recordsCount evaluation records through a bounded evidence-first AI operations workflow in the training lab.
Core 05 Evaluations and datasets Run a baseline evaluationRun a baseline evaluation through a bounded evidence-first AI operations workflow in the training lab.
Core 06 Evaluations and datasets Read aggregate scoresRead aggregate scores through a bounded evidence-first AI operations workflow in the training lab.
Core 07 Evaluations and datasets Inspect failed casesInspect failed cases through a bounded evidence-first AI operations workflow in the training lab.
Core 08 Evaluations and datasets Compare evaluation runsCompare evaluation runs through a bounded evidence-first AI operations workflow in the training lab.
Core 09 Evaluations and datasets Set a release thresholdSet a release threshold through a bounded evidence-first AI operations workflow in the training lab.
Core Checkpoint 10 Evaluations and datasets Review metric tradeoffsReview metric tradeoffs through a bounded evidence-first AI operations workflow in the training lab.
Core 11 Evaluations and datasets Inspect a data sliceInspect a data slice through a bounded evidence-first AI operations workflow in the training lab.
Core 12 Evaluations and datasets Find annotation disagreementFind annotation disagreement through a bounded evidence-first AI operations workflow in the training lab.
Core 13 Evaluations and datasets Detect evaluation leakageDetect evaluation leakage through a bounded evidence-first AI operations workflow in the training lab.
Admin 14 Evaluations and datasets Review dataset provenanceReview dataset provenance through a bounded evidence-first AI operations workflow in the training lab.
Admin 15 Evaluations and datasets Build a regression reportBuild a regression report through a bounded evidence-first AI operations workflow in the training lab.
Admin 16 Evaluations and datasets Make a release recommendationMake a release recommendation through a bounded evidence-first AI operations workflow in the training lab.
Admin 17 Evaluations and datasets Archive evaluation evidenceArchive evaluation evidence through a bounded evidence-first AI operations workflow in the training lab.
Admin 18 Evaluations and datasets Evaluations checkpointEvaluations checkpoint through a bounded evidence-first AI operations workflow in the training lab.
Admin Checkpoint
Save progress
Keep learning across sessions.
Create a free account to sync completed lessons, XP, streaks, course status, and your next lesson across sessions.
Checking account status...
Progress sync is idle.
0XP
New learnerrank
0day streak
0best streak
0daily streak
0lessons
0courses
0missions
0tasks
Noverified
Progress sync stores completion IDs, scores, streaks, badges, and game settings. Terminal contents, editor snippets, lab scratch, email, and real names are never published on leaderboards or share cards.
Command coverage
Commands you will practice here.
aiops prompts
Review versioned prompt metadata and deterministic prompt diffs.
aiops evals
Run and inspect bounded evaluation fixtures without model execution.
aiops rag
Inspect retrieval documents, chunks, index status, and reviewed results.
aiops traces
Read deterministic request traces, summaries, and latency evidence.
aiops guardrails
Review simulated guardrail policies, checks, and audit summaries.
aiops cost
Estimate cost and capacity from fixed training metrics.
aiops incidents
Review incident timelines, evidence bundles, and operator notes.
jq
Query JSON fixtures with a small declarative path subset; no expression execution occurs.