Troubleshooting · Lesson 23 of 26 · 8 minutes

Incident evidence bundle

Collect service, log, and network evidence in one drill.

Open the lab Course map

Mastery 0/3Run itExplain itApply it

learner@clairos:~ $

↑ history · Tab complete · Ctrl+C cancel · Esc, Tab to leaveExit 0

Your objectives

Hints are always available. Wrong turns never cost XP.

1
Check nginx
Run systemctl status nginx.
Hint & worked example
Inspect or change a simulated systemd service. Status, active state, and unit configuration reveal different evidence.
The command outcome must match the task, including successful access or the explicitly requested failure.
```
systemctl status nginx
```
Remember: A service can be active while an application endpoint is unhealthy; verify both layers.
2
Find 404
Run grep 404 /var/log/nginx-access.log.
Hint & worked example
Select input lines that match a pattern; -i ignores case, -n adds line numbers, and -v inverts selection.
The command outcome must match the task, including successful access or the explicitly requested failure.
```
grep 404 /var/log/nginx-access.log
```
Remember: No matches is exit status 1, which is different from a malformed invocation.
3
Capture listeners
Run ss -tulpn | tee lab/incident-listeners.txt.
Hint & worked example
Inspect simulated listening sockets and connection information. Copy a stream both to a file and standard output; -a appends.
The command outcome must match the task, including successful access or the explicitly requested failure.
```
ss -tulpn | tee lab/incident-listeners.txt
```
Remember: A listening socket is not proof of correct application responses or external reachability. tee can overwrite a file even though the output is also visible on screen.

Understand the pattern

Junior operators build confidence by gathering evidence before changing systems.

systemctl: Inspect or change a simulated systemd service. Status, active state, and unit configuration reveal different evidence.
A service can be active while an application endpoint is unhealthy; verify both layers.
grep: Select input lines that match a pattern; -i ignores case, -n adds line numbers, and -v inverts selection.
No matches is exit status 1, which is different from a malformed invocation.
ss: Inspect simulated listening sockets and connection information.
A listening socket is not proof of correct application responses or external reachability.
tee: Copy a stream both to a file and standard output; -a appends.
tee can overwrite a file even though the output is also visible on screen.

Worked example and expected result

$ systemctl status nginx

nginx.service - simulated nginx
   Loaded: loaded
   Active: active
   Notes: serving the simulated static learn site

Exit status: 0. The command completed successfully in the training fixture.

$ grep 404 /var/log/nginx-access.log

127.0.0.1 GET /api/test 404

Exit status: 0. The command completed successfully in the training fixture.

$ ss -tulpn | tee lab/incident-listeners.txt

Netid State  Local Address:Port  Process
udp   UNCONN 0.0.0.0:68          lab-dhcp
tcp   LISTEN 0.0.0.0:22          sshd
tcp   LISTEN 0.0.0.0:80          nginx
tcp   LISTEN 0.0.0.0:443         nginx

Exit status: 0. The command completed successfully in the training fixture.

Try a variation

Repeat this workflow in the sandbox with a different scratch filename. Keep the system fixtures unchanged and compare the observed output with your expectation before moving on.

Explain it

In “Incident evidence bundle”, which interpretation is safe?A successful first check guarantees that every dependent service is healthy.No matches is exit status 1, which is different from a malformed invocation.A command appearing in history proves it completed successfully.

Apply it elsewhere

Before adapting the grep workflow from “Incident evidence bundle” to another server, what should you do?Confirm the current directory, target identity, permissions, and the installed command behavior first.Paste the training paths unchanged into production.Skip output inspection whenever the command is familiar.

Keep the momentum.

Complete the lab, then explain and apply the pattern.

Next: Final triage loop Try an incident drill

Anonymous lesson and mastery progress lasts for this session. Sign in to keep synced lesson XP between sessions; mastery stays device-local. No speed bonus, lost lives, or hint penalties.

Keep learning

GNU Coreutils manual · GNU Bash manual