Home > other >  What is default behavior of Kubernetes when pod crashes?
What is default behavior of Kubernetes when pod crashes?

Time:02-14

In Kubernetes deployment with 4 static pods and no autoscaling, what happens by default if one pod crashes? Will it be re-created automatically with the same ID/different ID or will the application continue running on 3 pods?

CodePudding user response:

When a pod crashes, it will automatically be restarted. You will see this by the incrementing value of the pod's "Restarts" value when you do kubectl get pods

From the documentation: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#pod-template

Only a .spec.template.spec.restartPolicy equal to Always is allowed, which is the default if not specified.

In other words, a deployment will ALWAYS restart your pod, regardless, and you cannot change that behaviour.

A restart will not change the name of the pod (or ID has you have called it)

The only time the pod name will change is if the pod gets deleted. This can happen during autoscaling processes or if the pod gets evicted from a node.

You've specified no autoscaling in your deployment, but if you have specified a value of 4 replicas, as I suspect you have, then the eviction will cause that one pod to change names, as it gets recreated by another node, in order to meet your request for 4 replica.

By "changing names" I just mean the hash at the end of the pod name will change. So your pod named my-test-g4gsv may be renamed to my-test-4dsv4 after it goes to a new node.

There is a backoff policy for restarts. So if Kubernetes detects a pod has been restarted repeatedly, it will start delaying its restart attempts. You will notice this as a CrashLoopBackoff value under the pod status (instead of Running). While in this state, the pod is not started, so during this time, your deployment is essentially running with reduced replicas until Kubernetes starts it.

  • Related