Skip to content
Clairos Learn

Prompt and configuration operations · Lesson 13 of 14 · 6 min

Review prompt observability

Prompt observability links outcomes to a revision without routinely logging sensitive text. Revision IDs and bounded metrics make comparisons possible with less disclosure.

Open the labCourse map
Mastery 0/3Run itExplain itApply it
learner@clairosSimulated AI Ops lab
learner@clairos:~ $
↑ history · Tab complete · Ctrl+C cancel · Esc, Tab to leaveExit 0

Understand the pattern

Prompt observability links outcomes to a revision without routinely logging sensitive text. Revision IDs and bounded metrics make comparisons possible with less disclosure.

Interpret the evidence

Compare outcomes by p8 revision while keeping raw-prompt logging off.

A tempting mistake

You must publish prompt contents to identify which revision was used.

All names, measurements, prices, versions, and thresholds in this exercise are fictional training data. The evidence supports this bounded scenario, not a claim about a live provider.

Worked example and expected result
$ cat /opt/ai-lab/cases/aiops-prompts-config-013.json
{
  "fixture": "Fictional training data; not a live measurement.",
  "request_id": "req-31",
  "prompt_revision": "p8",
  "raw_prompt_logged": false,
  "quality_label": "supported"
}

Exit status: 0. The command completed successfully in the training fixture.

$ jq .prompt_revision /opt/ai-lab/cases/aiops-prompts-config-013.json
"p8"

Exit status: 0. The command completed successfully in the training fixture.

$ printf '%s\n' 'track-revision-outcomes' > /home/learner/lab/aiops-prompts-config-013-decision.txt
(no output; exit 0)

Exit status: 0. The command completed successfully in the training fixture.

Try a variation

Name the assumption that would have to change for “track-revision-outcomes” to stop being the justified action. Do not silently replace missing evidence with a guess.

Explain it

What does the evidence in “Review prompt observability” justify?

Apply it elsewhere

A colleague concludes: “You must publish prompt contents to identify which revision was used.” What should you do?

Keep the momentum.

Complete the lab, then explain and apply the pattern.

Next: Prompt operations checkpointTry an incident drill

Anonymous lesson and mastery progress lasts for this session. Sign in to keep synced lesson XP between sessions; mastery stays device-local. No speed bonus, lost lives, or hint penalties.

Keep learning

  1. 01Write a change noteA change note records why a revision exists, what evidence supports it and how to reverse it. Avoid vague success claims that cannot be checked against an evaluation.Practice
  2. 02Prompt operations checkpointA prompt release needs a reproducible manifest, passing evaluation and rollback. Syntax validity alone cannot show whether the new instruction achieves the intended task.Practice

Case methodology: evidence, limits, decision