Skip to content
Clairos Learn

Serving, containers, and resources · Lesson 12 of 17 · 8 min

Inspect GPU availability

GPU visibility and available memory are separate observations. Measure free memory and runtime attachment before deciding whether the next model can be loaded.

Open the labCourse map
Mastery 0/3Run itExplain itApply it
learner@clairosSimulated AI Ops lab
learner@clairos:~ $
↑ history · Tab complete · Ctrl+C cancel · Esc, Tab to leaveExit 0

Understand the pattern

GPU visibility and available memory are separate observations. Measure free memory and runtime attachment before deciding whether the next model can be loaded.

Interpret the evidence

Do not load a 6 GiB artifact into the observed 4 GiB of free memory without changing the plan.

A tempting mistake

Total GPU memory can be used again even when another process occupies it.

All names, measurements, prices, versions, and thresholds in this exercise are fictional training data. The evidence supports this bounded scenario, not a claim about a live provider.

Worked example and expected result
$ cat /opt/ai-lab/cases/aiops-serving-resources-012.json
{
  "fixture": "Fictional training data; not a live measurement.",
  "gpu_total_gib": 16,
  "gpu_used_gib": 12,
  "free_gib": 4,
  "new_model_gib": 6
}

Exit status: 0. The command completed successfully in the training fixture.

$ jq .free_gib /opt/ai-lab/cases/aiops-serving-resources-012.json
4

Exit status: 0. The command completed successfully in the training fixture.

$ printf '%s\n' 'hold-gpu-load' > /home/learner/lab/aiops-serving-resources-012-decision.txt
(no output; exit 0)

Exit status: 0. The command completed successfully in the training fixture.

Try a variation

Name the assumption that would have to change for “hold-gpu-load” to stop being the justified action. Do not silently replace missing evidence with a guess.

Explain it

What does the evidence in “Inspect GPU availability” justify?

Apply it elsewhere

A colleague concludes: “Total GPU memory can be used again even when another process occupies it.” What should you do?

Keep the momentum.

Complete the lab, then explain and apply the pattern.

Next: Match model memory to GPUTry an incident drill

Anonymous lesson and mastery progress lasts for this session. Sign in to keep synced lesson XP between sessions; mastery stays device-local. No speed bonus, lost lives, or hint penalties.

Keep learning

  1. 01Review a deployment diffRead a deployment diff for behavioral changes, not just additions and deletions. A replica change affects capacity and cost and should be reviewed against readiness and load.Practice
  2. 02Match model memory to GPUServing memory includes weights, cache and working buffers. Compare their combined demand with available memory, not only the artifact size.Practice
  3. 03Read readiness evidenceReadiness should cover the dependencies needed for useful work. A process can listen on a port while a required index or model remains unavailable.Practice

Case methodology: evidence, limits, decision