LearnDevOpsHands-On Labs
Troubleshooting Matrix/Pod stuck in Pending: 0/N nodes are available
KubernetesHigh Severity

Pod stuck in Pending: 0/N nodes are available

Fix a Pod stuck in Pending: run kubectl describe pod and read FailedScheduling. Fix Insufficient cpu, taints, volume conflicts, PVC binding and node selectors.

Observed Error Signature / Log Pattern
Warning  FailedScheduling  default-scheduler

Incident Overview

KubernetesKubernetes

Fix a Pod stuck in Pending: run kubectl describe pod and read FailedScheduling. Fix Insufficient cpu, taints, volume conflicts, PVC binding and node selectors.

medium fix·6 min read

By Vishvesh Patel·DevOps Engineer·Updated 16 September 2026

BASHDiagnostic / remediation command
Warning  FailedScheduling  default-scheduler
0/5 nodes are available: 2 Insufficient cpu, 2 node(s) had untolerated taint
{node-role.kubernetes.io/control-plane: }, 1 node(s) had volume node affinity conflict.

Do this first3 steps

Run these in order. Each one tells you what its output means before you change anything.

• Read why the scheduler refused, node by node

FailedScheduling gives a reason and a count per node: 'Insufficient cpu', 'had untolerated taint', 'had volume node affinity conflict'. The counts tell you which one applies to most of the cluster.

• If it is capacity, look at requests rather than usage

The scheduler counts requests, not actual consumption. A node sitting at 5% CPU can still be fully booked and correctly refuse your Pod.

• If it is taints, see which nodes carry them

Add a matching toleration to the Pod, or send it elsewhere. Control-plane and GPU nodes are tainted by default, so a small cluster can have fewer schedulable nodes than it appears.

A Pending Pod has been accepted by the API server but the scheduler cannot find a node for it. Unlike most Kubernetes failures, this one tells you exactly why, for every node.

The trick is knowing that the message is a summary of separate rejections, and the numbers add up to your node count.

BASHDiagnostic / remediation command
kubectl describe pod payments-api-7d4f-x8k2 | grep -A10 Events

Read the event

That reads as: five nodes, two rejected for CPU, two are control-plane nodes, one has a volume in the wrong zone. Every node has a reason. Fix the reason affecting the most nodes first.

BASHDiagnostic / remediation command
kubectl describe pod payments-api-7d4f-x8k2

Insufficient cpu / Insufficient memory

The most common, and the most misunderstood.

The scheduler compares your Pod's requests against each node's unreserved capacity, not against actual usage. A node sitting at 15% real CPU can be completely unschedulable because existing Pods have requested everything.

Limits above 100% is normal overcommit. Requests at 95% means the node is full, whatever kubectl top node says.

Three fixes, in order of preference:

Lower the Pod's requests if they exceed what the workload genuinely needs. Over-requesting is extremely common, since requests are usually guessed once and never revisited. Compare against real usage with kubectl top pod before assuming you need more nodes.

Right-size the existing Pods. If every node is 90% requested but 20% used, the cluster is not full. It is over-reserved. This is also where most cluster overspending lives.

Add capacity, via Cluster Autoscaler or Karpenter. Note that a Pod requesting more than any single node can provide will never schedule regardless of how many nodes exist:

BASHDiagnostic / remediation command
kubectl describe node ip-10-0-1-42 | grep -A8 "Allocated resources"

Untolerated taint

A taint marks a node as unsuitable unless a Pod explicitly tolerates it. Control-plane nodes carry one by default, which is why they appear in almost every one of these messages and can usually be ignored.

If a taint is blocking nodes you want to use, a GPU pool, a dedicated node group, add a toleration:

Watch for taints you did not add. node.kubernetes.io/disk-pressure, memory-pressure or not-ready are applied automatically by the kubelet and mean the node has a problem. The answer there is fixing the node, not tolerating it.

BASHDiagnostic / remediation command
kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints

Volume node affinity conflict

The Pod uses a PersistentVolumeClaim bound to a volume in one Availability Zone, and no suitable node exists in that zone.

Zonal block storage, EBS, GCP Persistent Disk, Azure Disk, can only attach within its own zone, so the volume pins the Pod there.

The root cause is nearly always a StorageClass using volumeBindingMode: Immediate, which provisions the disk before the scheduler has chosen a node. Switch to WaitForFirstConsumer, which defers provisioning until a node is picked so the disk lands in the right zone:

That fixes future volumes. An existing mismatched one must be migrated via a snapshot.

BASHDiagnostic / remediation command
kubectl get pv <pv-name> -o jsonpath='{.spec.nodeAffinity}'
kubectl get nodes -L topology.kubernetes.io/zone

Pod has unbound immediate PersistentVolumeClaims

Different message, different problem. The PVC itself has not bound.

If the event says waiting for first consumer to be created before binding, nothing is wrong, that is WaitForFirstConsumer working as designed, and it resolves once the Pod is scheduled. Otherwise the usual causes are no default StorageClass, no CSI driver installed, or an access mode the backing storage cannot provide (commonly ReadWriteMany on block storage).

BASHDiagnostic / remediation command
kubectl get pvc
kubectl describe pvc <name>

didn't match Pod's node affinity/selector

A nodeSelector or nodeAffinity rule matches no node. Usually a label typo, or a label that exists on no node at all:

BASHDiagnostic / remediation command
kubectl get nodes --show-labels
kubectl get pod <pod> -o jsonpath='{.spec.nodeSelector}'

didn't match pod anti-affinity rules

Anti-affinity requiring replicas on separate nodes cannot be satisfied because there are fewer eligible nodes than replicas. Three replicas with hard anti-affinity on a two-node cluster leaves one permanently Pending.

Either add nodes, or relax the rule from requiredDuringSchedulingIgnoredDuringExecution to preferredDuringSchedulingIgnoredDuringExecution, which makes it best-effort.

This is also why single-node clusters are misleading for testing. The rule silently does nothing there and then blocks scheduling in production.

Too many pods

Nodes cap how many Pods they run. On EKS with the VPC CNI this is driven by ENI limits, so a t3.medium tops out at 17 Pods regardless of how idle it looks. Enabling prefix delegation raises it substantially.

BASHDiagnostic / remediation command
1 node(s) had reached the max pod limit

A checklist

Frequently Asked Questions

What does "0/5 nodes are available" mean?

The scheduler evaluated all five nodes and rejected every one, and the rest of the message states why for each. The numbers sum to your node count, so 2 Insufficient cpu, 2 untolerated taint, 1 volume node affinity conflict accounts for all five. Treat it as a checklist rather than a single error: fix whichever reason affects the most nodes first, since that is the one standing between you and a schedulable Pod.

Why does Kubernetes say Insufficient cpu when my nodes look idle?

Because scheduling works from the sum of Pod requests on each node, never from measured usage. A node running at 15% actual CPU can be fully committed if the Pods on it requested all its allocatable capacity. Run kubectl describe node <name> and read the "Allocated resources" section, which shows requests as a percentage of capacity. Requests far above real usage are extremely common and are also where most cluster overspending comes from.

What is a volume node affinity conflict?

The Pod needs a PersistentVolumeClaim bound to a volume in one Availability Zone, and the scheduler cannot place the Pod on a node in that zone. Zonal block storage such as EBS can only attach within its own zone, so the volume effectively pins the Pod. The root cause is usually a StorageClass with volumeBindingMode: Immediate, which creates the disk before a node is chosen. Switching to WaitForFirstConsumer prevents it for new volumes; an existing mismatched volume needs migrating via a snapshot.

My PVC says "waiting for first consumer". Is that an error?

No. It is volumeBindingMode: WaitForFirstConsumer behaving exactly as intended. The StorageClass deliberately defers provisioning until the scheduler picks a node, so the disk is created in the right Availability Zone rather than an arbitrary one. It binds as soon as a Pod that uses it is scheduled. Genuine PVC problems produce different messages, typically about no available persistent volumes or no default StorageClass.

Why is one replica always Pending when the others are running?

Usually pod anti-affinity requiring replicas on distinct nodes, with fewer eligible nodes than replicas. Three replicas with a hard anti-affinity rule on a two-node cluster leaves one permanently unschedulable. Either add nodes, or change requiredDuringSchedulingIgnoredDuringExecution to preferredDuringSchedulingIgnoredDuringExecution so the rule becomes best-effort. The other candidate is a ReadWriteOnce volume already attached to a different node.

How do I fix "node(s) had reached the max pod limit"?

Nodes have a cap on concurrent Pods, visible via kubectl get nodes -o custom-columns=NAME:.metadata.name,MAXPODS:.status.allocatable.pods. On EKS with the AWS VPC CNI that limit comes from how many IP addresses the instance's network interfaces can carry. A t3.medium allows only 17 Pods regardless of its CPU and memory being idle. Enable prefix delegation on the CNI, which assigns /28 blocks per interface and raises the ceiling substantially, or use larger instance types.

Reference and practice

Learn the underlying concept

Other Kubernetes errors

Free, comprehensive DevOps tutorials and learning roadmaps. Master Docker, Kubernetes, CI/CD, and more.

Get notified about new tutorials and features.

• Roadmaps

• Interview Prep

• Certifications

• Company

© 2026 DevOpsLesson. All rights reserved.

Need to practice this scenario?

Simulate this error condition and practice recovery commands safely in our in-browser terminal.

Open Kubernetes Sandbox