Skip to content

Room 38

EVENTS & TROUBLESHOOTING

The Black Box

  • Every action in K8s generates events. They’re the flight recorder of your cluster.
  • A deployment is failing. Use events to diagnose and fix it.

📜 Mission Briefing

  1. Apply broken-deploy.yaml - pods will fail to start.
  2. Use kubectl get events --sort-by=.metadata.creationTimestamp to see what happened.
  3. The events reveal the issue: wrong image name.
  4. Fix the deployment and get the password from the running pod.
  5. The password is eventlog.
Key Commands
Command Purpose
kubectl get events List events
kubectl get events --sort-by=.metadata.creationTimestamp Sorted events
kubectl get events -n NS --field-selector reason=Failed Filter events
kubectl describe pod NAME Events section
Troubleshooting Workflow
# 1. Check pod status
kubectl get pods
# Common states: Pending, Running, CrashLoopBackOff, ImagePullBackOff, Error

# 2. Describe the pod (most useful)
kubectl describe pod my-pod
# Look at:
#   - Conditions
#   - Events (bottom section)
#   - Container statuses

# 3. Check events
kubectl get events --sort-by='.lastTimestamp'
kubectl get events --field-selector type=Warning
kubectl get events --field-selector involvedObject.name=my-pod

# 4. Check logs
kubectl logs my-pod
kubectl logs my-pod --previous  # if CrashLoopBackOff

# 5. Common issues:
# ImagePullBackOff:  Wrong image name or registry auth
# CrashLoopBackOff:  Container crashing repeatedly
# Pending:           Insufficient resources or scheduling issues
# OOMKilled:         Out of memory
# CreateContainerConfigError: Bad ConfigMap/Secret reference

# 6. Node-level issues
kubectl describe node worker-1
kubectl get nodes -o wide
Hints

Hint 1

kubectl apply -f broken-deploy.yaml
kubectl get events --sort-by=.metadata.creationTimestamp

Hint 2

Look for Failed or Warning events. Fix the image name and re-apply.

Hint 3

The password is eventlog.

🔓 Unlock Room 39

openssl enc -aes-256-cbc -d -a -pbkdf2 \
        -in ../room_39/README -out ../room_39/README.txt -pass pass:PASSWORD
cat ../room_39/README.txt