← All posts
Infra

Kubernetes ImagePullBackOff, admission denied, or FailedScheduling? Read the Pod events, not the status

For developers and autonomous AI agents · ~6 min read

A pod is stuck. The STATUS says ImagePullBackOff or CreateContainerConfigError, or an event says admission webhook "..." denied the request. You re-apply the deployment. Nothing changes. The mistake is reading the status label as the diagnosis — in Kubernetes, the status is a state, and the real answer is in the Pod events.

ImagePullBackOff: a backoff state, not an error to retry

ErrImagePull is the initial pull failure; ImagePullBackOff is the exponential-backoff wait state that follows (capping at 5 minutes). Critically, the container never started, so there are no application logskubectl logs fails with "waiting to start: trying and failing to pull image."

The actionable string lives ONLY in the Pod events: kubectl describe pod <p> -n <ns> → Events → "Failed to pull image". Map it: manifest unknown/not found = wrong tag/digest (don't assume :latest exists); pull access denied / unauthorized = a missing or expired imagePullSecret (create a kubernetes.io/dockerconfigjson secret in the SAME namespace and link it to the ServiceAccount, not just the pod); 429 Too Many Requests = a registry rate-limit. Fix it, then delete the pod to skip the backoff timer.

Admission webhook denied: terminal, fix the manifest

admission webhook "..." denied the request: <reason> means a validating webhook or policy engine (OPA-Gatekeeper, Kyverno) rejected the resource. This is terminal — the apiServer rejects the identical manifest on every retry. The denial reason says exactly what to fix ("must specify resource limits", "image must be signed", "must use a digest"). If instead you see failed calling webhook ... connection refused, the webhook backend is down — and with failurePolicy: Fail it blocks every matching resource cluster-wide.

FailedScheduling, Evicted, CreateContainerConfigError

For agents deploying to Kubernetes

An agent that treats these states as retryable loops forever. diagnose_infra_error returns the kubernetes family and points at the right fix — read the event, not the status:

diagnose_infra_error({error: "imagepullbackoff manifest unknown failed to pull image"})
// → { family: "kubernetes", action_class: "config",
//      fix: "read the Pod event; fix the tag/imagePullSecret; delete the pod to skip the backoff" }
For AI agents

Machine-readable guide: https://snapback.sh/llms.txt · Tool discovery: https://snapback.sh/.well-known/mcp.json · MCP endpoint: https://api.snapback.sh/mcp. Free tools need no token; pay-per-call diagnosis via x402 on Solana or EVM (~$0.0032 USDC).

Try Snapback freeRead the docs