Understanding Pod Lifecycle in Kubernetes

Last updated: January 7, 2024

1. Overview

In this tutorial, we’ll dive deep into the Kubernetes pod lifecycle. A pod is a self-contained unit. It holds one or more related containers and associated resources and represents a single instance of an application.

2. Understanding the Pod Lifecycle

Let’s detail every step of the pod’s life from its creation till its termination.

2.1. Pod Creation

Pods are amongst Kubernetes base objects. Thus, to create a pod, we need to define its configuration as a JSON or YAML file. The configuration describes the desired state of the pod, for example:

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
    - name: mycontainer
      image: nginx:latest

Then, we can use Kubernetes’ command-line tool kubectl to apply the configuration and create the pod:

$ kubectl apply -f pod.yaml

We can now check that the creation went well, for instance, by listing all the pods:

$ kubectl get pods
NAME    READY   STATUS              RESTARTS   AGE
mypod   0/1     ContainerCreating   0          5s

After a couple of seconds more, we can notice the status change:

$ kubectl get pods 
NAME    READY   STATUS    RESTARTS   AGE 
mypod   0/1     Running   0          13s

2.2. Scheduling

Scheduling is the process by which Kubernetes assigns a node in the cluster to run the pod. There are several ways to influence the assignment:

  • Using nodeSelector to constrain the pod to nodes with specific labels
  • Using affinity and anti-affinity to specify more complex constraints
  • Tainting nodes to refuse pods unless they have a certain toleration
  • Setting pod priority
  • Indicating the amount of CPU and memory required by the pod

Once the pod is scheduled to the best node, it remains there until termination. If the node dies, Kubernetes also deletes the pod. To manage the available pod instances, we should use controllers.

2.3. Initialization

The initialization phase ensures the pod is fully prepared to execute its workload. First, if there is any init container in the pod specification, Kubernetes will execute it. This setup phase then enables the execution of the main application containers. The so-called container runtime is responsible for running containers. When the execution of all containers is completed successfully, the pod is effectively ready. The pod readiness ensures that he can receive and serve traffic.

2.4. Running the Pod: Container States and Restart Policy

Kubernetes monitors the state of each container inside a pod. Each container has three possible states:

  • Running: the container is executing without issues
  • Terminated: the container completed is task or failed. In this case, a kubectl query will give the reason, exit code, begin and end dates of the container execution
  • Waiting: a temporary state for a container that isn’t fully started or terminated

Let’s check the state of our pod:

$ kubectl describe pod mypod
(...)
Containers:
  mycontainer:
    State: Running

Furthermore, we can use restart policies to bring more resilience to our Kubernetes environment. A restart policy dictates the pod’s response when a container exits. There are three possible restart policies:

  • Always: Kubernetes immediately restarts the container. It is the default value
  • OnFailure: Kubernetes restarts the container only if it exited because of a failure
  • Never: no restart on exit

These policies apply to app containers and regular init containers in the pod. However, sidecar containers ignore the pod-level restartPolicy.

2.5. Termination

The pod termination process ensures processes can terminate gracefully instead of being killed savagely with no chance to clean up. To request a pod deletion, we use the delete command:

$ kubectl delete pod mypod
pod "mypod" deleted

On deletion, kubelet, the node-level Kubernetes agent, attempts a graceful pod shutdown. It sends a message to the pod for it to shut down its containers. The default grace period for termination is 30 seconds, but it can be overriden via the terminationGracePeriodSeconds attribute. Setting this period to 0 forces the immediate deletion of the pod.

In any other cases, the pod will run its preStop hooks. Then, it will stop the application containers. Lastly, the sidecar containers terminate in the reverse order of their definition. However, if a problem occurs during the graceful termination period, the kubelet may eventually force the termination of the pod.

After termination, Kubernetes performs garbage collection to reclaim the freed resources. This includes releasing IP addresses, removing associated volumes, and updating relevant status.

3. Exploring PodStatus Objects

The PodStatus object provides real-time information about the state of a pod. Understanding and monitoring PodStatus is crucial for maintaining awareness of a pod’s health and status within a cluster. In particular:

  • PodConditions help understand the overall state of the pod
  • ContainerStatus provides detailed information about the state of each container
  • The pod phase is a summary of where the pod is in its lifecycle

We can obtain the PodStatus of our pod via the kubectl get command:

$ kubectl get pod mypod -o jsonpath='{.status}'
{“conditions”:[{“lastProbeTime”:null,“lastTransitionTime”:"2023-12-29T17:40:21Z",“status”:“True”,“type”:“Initialized”},{“lastProbeTime”:null,“lastTransitionTime”:"2023-12-29T17:40:24Z",“status”:“True”,“type”:“Ready”},{“lastProbeTime”:null,“lastTransitionTime”:"2023-12-29T17:40:24Z",“status”:“True”,“type”:“ContainersReady”},{“lastProbeTime”:null,“lastTransitionTime”:"2023-12-29T17:40:21Z",“status”:“True”,“type”:“PodScheduled”}],“containerStatuses”:[{“containerID”:"docker://93b9221086d2b61f6987828ccf84a1242ce3bf981a77b570cb558990332d87ec",“image”:"nginx:latest",“imageID”:"docker-pullable://nginx@sha256:2bdc49f2f8ae8d8dc50ed00f2ee56d00385c6f8bc8a8b320d0a294d9e3b49026",“lastState”:{},“name”:“mycontainer”,“ready”:true,“restartCount”:0,“started”:true,“state”:{“running”:{“startedAt”:"2023-12-29T17:40:24Z"}}}],“hostIP”:"192.168.65.4",“phase”:“Running”,“podIP”:"10.1.0.12",“podIPs”:[{“ip”:"10.1.0.12"}],“qosClass”:“Burstable”,“startTime”:"2023-12-29T17:40:21Z"}

4. Leveraging Probes

Probes are a mechanism for assessing the health and readiness of containers within a pod. Their role is crucial in ensuring that applications are running as expected and ready to handle traffic.

4.1. Liveness Probe

Liveness probes make Kubernetes periodically check the container’s health. If the probe fails, Kubernetes restarts the container to restore its health:

livenessProbe:
  httpGet:
    path: /health
    port: 80
  initialDelaySeconds: 5
  periodSeconds: 10

In this example, the liveness probe checks the /health path on port 80 of the container every 10 seconds, starting 5 seconds after the container starts. Additionally, the failureThreshold parameter, which defaults to 3, sets the number of consecutive failures to tolerate before restarting the container.

4.2. Readiness Probes

Readiness probes determine if a container is ready to handle incoming traffic. When a container fails its readiness probe, Kubernetes stops directing traffic to that container until it passes the probe.

4.3. Startup Probes

Startup probes indicate whether the container application has started. All other probes are disabled until the startup probe succeeds. This way, we can keep short liveness intervals to test our pods while avoiding deadlocks on startup.

5. Monitoring With PodConditions

Pod conditions are a set of status indicators that provide critical information about the health and state of a pod. As we saw, PodConditions are part of the PodStatus object. The key conditions are:

  • PodScheduled: indicates whether Kubernetes assigned a node to the pod
  • ContainersReady: true if every container is up and running, false if at least one container is still initializing or having issues
  • Initialized: focuses on the init containers’ completion
  • Ready: determines if the pod is ready to serve requests

6. Tracking the Pod Lifecycle with Events

Events provide a valuable way to track and monitor the entire lifecycle of a pod. They provide a detailed chronological log of activities and state changes within a cluster. Let’s view our pod events:

$ kubectl get events
LAST SEEN   TYPE     REASON      OBJECT      MESSAGE
28m         Normal   Scheduled   pod/mypod   Successfully assigned default/mypod to docker-desktop
26m         Normal   Pulling     pod/mypod   Pulling image "nginx:latest"
28m         Normal   Pulled      pod/mypod   Successfully pulled image "nginx:latest" in 1.22447429s (1.22448973s including waiting)
26m         Normal   Created     pod/mypod   Created container mycontainer
26m         Normal   Started     pod/mypod   Started container mycontainer

In particular, Kubernetes logs scheduling events, container state changes, error conditions, and application-specific events.

7. Conclusion

In this article, we studied the entire pod lifecycle from creation until termination. However, getting our pod live on production isn’t the end of the story. That’s why we also introduced ways to monitor our pod’s state. The Kubernetes ecosystem is still evolving dynamically, so we can recommend further exploration of any necessary topic.

Comments are open for 30 days after publishing a post. For any issues past this date, use the Contact form on the site.