What Happens When Kubernetes Pods Disappear? A Practical Guide to Pod Disruption Budgets

You’re in the middle of the holiday season. Your e-commerce platform is running smoothly when your DevOps team schedules node maintenance. Suddenly, key parts of your system become unavailable because Kubernetes terminated multiple pods simultaneously. Your customers see errors, transactions fail, and revenue plummets.

Or imagine your organization introduces an autoscaling policy to save costs. During scale-down events, all pods of a critical microservice get removed, leading to partial application outages.

These situations highlight the chaos that can occur without a Pod Disruption Budget (PDB). PDBs ensure your applications remain resilient and available, even during planned disruptions like node maintenance, autoscaling, or cluster upgrades.

In this comprehensive guide, we’ll explore PDBs, how they work, and how to implement them step-by-step in a real Kubernetes cluster.

What is a Pod Disruption Budget (PDB)?

A Pod Disruption Budget (PDB) is a Kubernetes object that limits the number of pods in a deployment, stateful set, or replica set that can be voluntarily disrupted at any given time.

Disruptions could be caused by:

• Node maintenance or upgrades
• Cluster autoscaling (scale-down events)
• Admin-triggered operations like kubectl drain

PDBs guarantee a minimum number of running pods (or restrict the maximum number of disrupted pods) during such operations.

Key PDB Parameters

1. minAvailable: The minimum number of pods that must remain running during disruptions.
2. maxUnavailable: The maximum number of pods that can be unavailable during disruptions.

Note: PDBs are not applicable for involuntary disruptions, such as node crashes or unplanned outages.

How Does a PDB Work in Kubernetes?

Kubernetes relies on the PDB to evaluate whether a disruption is permissible. If the disruption would cause the remaining number of pods to fall below the defined threshold, Kubernetes blocks the action.

For example:

• You have 5 replicas of a pod and set minAvailable: 4. If an operation tries to disrupt 2 pods, Kubernetes will block it because only 3 pods would remain, violating the PDB.

Common Scenarios Without PDBs

1. Cluster Upgrades: During node updates, Kubernetes drains nodes (evicting pods). Without a PDB, critical applications might lose all pods on a node.
2. Autoscaling Chaos: Scaling down without a PDB can unintentionally remove all replicas of a critical service.
3. Load Balancer Failures: If all replicas of a backend service are disrupted, load balancers might start serving errors.

Step-by-Step Implementation of Pod Disruption Budgets

Here’s how to implement a Pod Disruption Budget in a real Kubernetes cluster:

1. Set Up Your Kubernetes Cluster

Ensure you have a running Kubernetes cluster. You can use:

• Minikube: For local testing
• EKS, AKS, or GKE: For cloud-based clusters
• KIND: Kubernetes in Docker

Connect to your cluster using kubectl. Verify your cluster is running:

kubectl get nodes

2. Deploy a Sample Application

Let’s deploy a simple Nginx application:

apiVersion: apps/v1  
kind: Deployment  
metadata:  
  name: nginx-deployment  
  labels:  
    app: nginx  
spec:  
  replicas: 5  
  selector:  
    matchLabels:  
      app: nginx  
  template:  
    metadata:  
      labels:  
        app: nginx  
    spec:  
      containers:  
      - name: nginx  
        image: nginx:latest  
        ports:  
        - containerPort: 80

Apply the deployment:

kubectl apply -f nginx-deployment.yaml

Check the pods:

kubectl get pods -l app=nginx

3. Create a Pod Disruption Budget

To ensure at least 4 pods are always running:

apiVersion: policy/v1  
kind: PodDisruptionBudget  
metadata:  
  name: nginx-pdb  
spec:  
  minAvailable: 4  
  selector:  
    matchLabels:  
      app: nginx

Apply the PDB:

kubectl apply -f nginx-pdb.yaml

Verify the PDB:

kubectl get pdb nginx-pdb

4. Simulate a Disruption

Drain a node to simulate voluntary disruption:

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

Kubernetes will respect the PDB and prevent disruptions that violate it.

5. Monitor Your PDB

Use the following command to monitor the status:

kubectl describe pdb nginx-pdb

Look for fields like:

• Allowed Disruptions
• Current Healthy
• Desired Healthy

6. Test Cluster Autoscaling (Optional)

If you’re using a cloud provider, scale down a node group to test how PDB interacts with autoscaling:

aws eks update-nodegroup-config --cluster-name <cluster> --nodegroup-name <group> --scaling-config minSize=1,maxSize=5,desiredSize=2

Real-World Best Practices

1. Start Small: Test PDBs on non-critical workloads before applying them to production services.
2. Balance Constraints: Avoid setting minAvailable too high—it might block scaling or upgrades.
3. Use with HPA: Combine PDB with the Horizontal Pod Autoscaler to dynamically scale pods while ensuring availability.
4. Review Regularly: Update PDBs as traffic patterns or resource requirements change.
5. Monitor Continuously: Use monitoring tools like Prometheus, Grafana, or Lens to visualize PDB health.

What Happens If You Skip PDBs?

Without PDBs, you risk:

• Outages during maintenance: Losing all replicas of a service.
• Failed scaling operations: Removing critical pods unintentionally.
• Poor user experience: Load balancers returning 503 errors due to missing pods.

Final Thoughts

Kubernetes Pod Disruption Budgets are an essential tool for maintaining application uptime during planned disruptions. By defining clear thresholds for pod availability, you can:

• Minimize downtime
• Protect critical workloads
• Enable smoother cluster operations

Start small, monitor continuously, and iterate as your workloads evolve. With a properly configured PDB, your systems will stay resilient even in the face of voluntary disruptions.