| Myth | Reality |
|---|---|
| Kubernetes bills scale mainly with traffic growth | Cast AI’s benchmark data across thousands of clusters shows the dominant cost driver is idle, over-requested CPU and memory, not traffic volume |
| Spot instances are too risky for production Kubernetes workloads | With pod disruption budgets, multi-AZ spread, and diversified node provisioning, spot nodes routinely run stateless production workloads at 60-90 percent lower node cost |
| Setting generous CPU and memory requests is the safe default | Generous requests are the single largest source of waste; Cast AI’s 2026 report found average provisioned CPU utilization has fallen to about 8 percent |
| The cluster autoscaler alone handles bin-packing | Autoscalers mainly add nodes for pending pods; separate consolidation tooling is what compacts already-running workloads and removes emptied nodes |
| Cloud billing dashboards already give full Kubernetes cost visibility | Billing dashboards show total node spend, not per-namespace or per-team allocation, which is why Kubernetes cost tooling remains a distinct FinOps discipline |
Why Kubernetes Costs Behave Differently From Ordinary Cloud Costs
A standard cloud bill maps fairly directly to what a team provisioned: one VM, one bill line. Kubernetes breaks that mapping. A cluster’s node fleet is billed as a pool of compute, while the actual consumers of that pool, namely pods, request resources independently of what they use, get rescheduled across nodes constantly, and are rarely tied cleanly back to a cost center. This is why Kubernetes cost optimization is treated as its own discipline rather than a subset of general cloud cost management: the waste hides inside the scheduling layer, not on a line item anyone reviews monthly.
The scale of the gap is well documented. Cast AI’s cost benchmark research, drawn from thousands of clusters running across AWS, Google Cloud, and Azure, has tracked this consistently for several years running, and the direction has gotten worse, not better, even as tooling has matured.
The Data: How Much Kubernetes Capacity Actually Goes to Waste
| Metric | Reported figure | Source |
|---|---|---|
| Average CPU utilization of provisioned capacity | About 8 percent, down from 10 percent the prior year | Cast AI, 2026 State of Kubernetes Optimization Report |
| Average memory utilization of provisioned capacity | Roughly 20 to 23 percent | Cast AI, 2025-2026 Kubernetes Cost Benchmark research |
| CPU utilization on clusters with 50 or more CPUs | Only about 13 percent of provisioned CPU actually used | Cast AI Kubernetes Cost Benchmark Report |
| Year-over-year change in CPU overprovisioning | Widened from roughly 40 percent to roughly 69 percent | Cast AI, 2026 State of Kubernetes Optimization Report |
| Organizations running Kubernetes in production | 82 percent of container users, up from 66 percent in 2023 | CNCF, 2025 Annual Cloud Native Survey |
Figure 1: The Three Layers of Kubernetes Resource Waste
A three-tier bar chart. The top bar represents “provisioned” node capacity at 100 percent, the amount actually paid for. The middle bar represents “requested” resources, the amount a pod’s spec declares it needs, typically 40-70 percent of provisioned capacity once accounting for headroom and safety margins added by developers. The bottom bar represents “actually used” resources, measured at real-time CPU and memory consumption, which per Cast AI’s benchmark data lands around 8-10 percent of the original provisioned figure for CPU. The gap between each layer is where cost optimization work happens: right-sizing closes the requested-to-used gap, and bin-packing closes the provisioned-to-requested gap.
Right-Sizing: Closing the Requests-vs-Usage Gap
Vertical Pod Autoscaler
The Vertical Pod Autoscaler, documented directly in the Kubernetes project, watches a pod’s actual CPU and memory consumption over time and recommends, or in some modes automatically applies, adjusted resource requests. Running VPA in recommendation-only mode first is the safer rollout path: it surfaces the gap between what a deployment requests and what it consumes without touching anything, giving a platform team a concrete, per-workload right-sizing backlog instead of a blanket policy.
Horizontal Pod Autoscaler and When Not to Combine Them
The Horizontal Pod Autoscaler scales the number of pod replicas up or down based on observed metrics, most commonly CPU or memory utilization, or custom application metrics. HPA and VPA can conflict when both are configured against the same metric on the same workload, since one tries to add replicas while the other tries to resize them, producing oscillation rather than stability. The Kubernetes documentation explicitly flags this interaction; the practical fix is scaling horizontally on request-driven custom metrics while letting VPA own CPU and memory sizing, or restricting VPA to recommendation-only mode on any workload that also has an HPA attached.
Bin-Packing: Paying for Nodes You Actually Use
How the Default Scheduler Leaves Capacity Stranded
The default Kubernetes scheduler places new pods onto nodes with available capacity, but it has no ongoing incentive to consolidate already-running pods onto fewer nodes once earlier scheduling decisions leave fragments of unused capacity scattered across a fleet. A cluster can easily end up running twelve nodes at 25 percent utilization each when the same workload would fit comfortably on four nodes at 75 percent utilization, and the difference is pure wasted node-hours.
Consolidation Tooling
This is the specific problem that bin-packing and consolidation tools solve. Karpenter, AWS’s open-source cluster autoscaling project, continuously evaluates whether existing workloads could be repacked onto fewer or cheaper nodes and actively drains and terminates nodes once they are no longer needed, rather than just adding capacity for pending pods. The Kubernetes descheduler project takes a complementary approach, evicting pods that violate scheduling best practices, such as being poorly spread across zones or stuck on an underutilized node, so the scheduler gets another chance to place them efficiently. Neither replaces right-sizing: a cluster full of over-requested pods will still bin-pack poorly, because bin-packing operates on whatever the requests say, not on real usage.
Spot and Preemptible Instances: The Discount Layer
| Provider | Product name | Maximum discount vs. on-demand | Source |
|---|---|---|---|
| AWS | EC2 Spot Instances | Up to 90 percent | AWS, official EC2 Spot Instances pricing page |
| Google Cloud | Spot VMs | Up to 91 percent | Google Cloud, Compute Engine Spot VM documentation |
| Azure | Spot Virtual Machines | Up to 90 percent | Microsoft Azure, Spot Virtual Machines pricing page |
Spot and preemptible capacity is spare cloud inventory sold at a steep discount, reclaimable by the provider on short notice, typically with a two-minute warning on AWS and Azure. This makes it a poor fit for stateful, single-replica workloads but an excellent fit for anything that is stateless, horizontally replicated, and tolerant of a pod being rescheduled elsewhere within seconds. Batch jobs, CI runners, and horizontally scaled stateless API replicas behind a load balancer are the textbook use cases, provided PodDisruptionBudgets are set so the cluster never evicts too many replicas of the same service at once, and workloads are spread across multiple instance types and availability zones so a single spot-market shortage does not take down an entire deployment simultaneously. For workloads with heavier, less interruption-tolerant compute needs, such as GPU-backed training jobs, the same rent-versus-own trade-off shows up again at a larger scale, covered in this site’s TCO comparison of renting versus owning GPU capacity.
Figure 2: Cumulative Effect of Stacking Cost Levers
A waterfall chart starting at a baseline monthly Kubernetes node bill of 100 percent. Right-sizing resource requests to match observed usage, informed by VPA recommendations, typically removes a large first slice of unnecessary reserved headroom. Bin-packing and node consolidation remove a further slice by eliminating stranded capacity spread across too many nodes. Moving eligible stateless and batch workloads to spot or preemptible instances removes the largest remaining slice, since the discount applies to whatever compute footprint is left after the first two steps. The order matters: applying spot discounts before right-sizing simply buys a steep discount on capacity that was already wasted.
Putting It Together: An Order of Operations That Compounds Savings
Teams that treat these three levers as a menu to pick from tend to under-perform teams that treat them as a sequence. Right-sizing first, so requests reflect real usage. Bin-packing and consolidation second, so the now-accurate requests are packed efficiently onto the fewest nodes. Spot and preemptible capacity last, applied to whatever compute footprint remains once the first two steps have removed the waste that a discount would otherwise just be masking. Skipping straight to spot instances on an over-provisioned cluster still saves money, but it saves a percentage of a number that should have been much smaller to begin with. This same discipline of sequencing the highest-leverage fix first, rather than the most visible one, mirrors the workload-tiering approach covered in this site’s look at cloud repatriation and its real cost trade-offs.
Common Mistakes in Kubernetes Cost Optimization
- Setting resource limits far above requests “just in case,” which defeats bin-packing since the scheduler reasons primarily about requests, not limits.
- Running VPA in auto-apply mode on a workload that also has an HPA configured against the same metric, causing oscillating restarts instead of stable sizing.
- Moving every workload to spot capacity indiscriminately, including stateful, single-replica services that cannot tolerate sudden termination.
- Optimizing compute aggressively while ignoring cross-zone and cross-region data transfer, which can erase node savings entirely on chatty, distributed workloads.
- Treating a one-time right-sizing pass as finished, when workload resource needs drift within weeks as code and traffic patterns change.
In Practice
Platform teams that sustain Kubernetes savings over time, rather than seeing costs creep back up within a quarter, tend to build the three levers into automated, continuous processes instead of one-off projects. Right-sizing runs on a schedule, with VPA recommendations reviewed and rolled out to lower-risk namespaces first, rather than a single manual pass across the whole cluster. Bin-packing tools run continuously rather than being triggered by hand, since workload footprints change every deploy. Spot adoption is expanded incrementally, starting with the most interruption-tolerant batch and CI workloads before extending to production-facing stateless services once disruption budgets and multi-AZ spread are proven reliable. And cost data is attributed back to namespaces and teams so that whoever owns a workload sees its actual resource waste directly, closing the same visibility gap that shows up in general cloud spend management, discussed further in this site’s piece on architecting around egress fees and data gravity.
When Kubernetes Cost Tooling Is Worth Adopting Versus Doing It Manually
Manual right-sizing and manual node consolidation are realistic for a single cluster with a small, stable set of workloads, using nothing more than the Kubernetes Metrics Server and VPA in recommendation mode. Past roughly a few dozen workloads or several clusters, the constant drift in what each workload actually needs outpaces what a team can track by hand, which is when dedicated Kubernetes cost and automation platforms earn their subscription cost. Teams evaluating whether to run this infrastructure themselves at all, rather than on a hyperscaler, face a related but distinct calculation covered in this site’s comparison of bare-metal and neocloud alternatives to hyperscalers, and teams unsure whether they need a cluster at all should start with the more basic question addressed in this site’s guide to honest Kubernetes alternatives for smaller teams.
Glossary
- Resource request
- The amount of CPU or memory a container declares it needs, used by the scheduler to decide which node can host it.
- Resource limit
- The maximum CPU or memory a container is allowed to use, enforced at the node level, distinct from what the scheduler reasons about when placing pods.
- Vertical Pod Autoscaler (VPA)
- A Kubernetes component that recommends or automatically applies adjusted resource requests based on a workload’s observed usage history.
- Horizontal Pod Autoscaler (HPA)
- A Kubernetes component that scales the number of pod replicas up or down based on observed metrics such as CPU utilization or custom signals.
- Bin-packing
- Scheduling workloads onto the fewest possible nodes to minimize stranded, idle capacity spread across a fleet.
- Node consolidation
- The process of repacking running workloads onto fewer nodes and terminating emptied nodes, performed continuously by tools like Karpenter.
- Spot instance
- Spare cloud compute capacity sold at a steep discount, reclaimable by the provider with short notice, suited to stateless and interruption-tolerant workloads.
- PodDisruptionBudget
- A Kubernetes policy that limits how many replicas of a workload can be voluntarily evicted at once, protecting availability during node consolidation or spot reclamation.
Key Takeaways
- Kubernetes cost waste hides in the scheduling layer, not on a bill line item, which is why it needs its own optimization discipline separate from general cloud cost management.
- Cast AI’s benchmark research puts average provisioned CPU utilization at roughly 8 to 10 percent, with the overprovisioning gap widening year over year rather than shrinking.
- Right-sizing with the Vertical Pod Autoscaler should generally run before bin-packing, since packing efficiency is meaningless if requests do not reflect real usage.
- Running VPA and HPA against the same metric on one workload commonly causes scaling oscillation; splitting responsibility between the two avoids it.
- Bin-packing and node consolidation tools like Karpenter reclaim stranded capacity that the default scheduler and cluster autoscaler leave behind.
- Spot and preemptible instances offer discounts of up to 90 to 91 percent across AWS, Google Cloud, and Azure, but should be applied after right-sizing and bin-packing, not before.
- Sustainable savings come from running all three levers as continuous, automated processes with per-team cost visibility, not a one-time optimization project.
FAQs
What is the biggest driver of wasted Kubernetes spend?
Over-requested CPU and memory is consistently the largest driver, not traffic growth or node pricing. Cast AI’s benchmark data shows average provisioned CPU utilization sitting around 8 to 10 percent, meaning the overwhelming majority of paid-for compute capacity across a typical cluster sits idle at any given moment.
Should I use the Vertical Pod Autoscaler or the Horizontal Pod Autoscaler first?
Start with VPA in recommendation-only mode to understand real resource usage before changing anything. Once requests are right-sized, add HPA for workloads with variable traffic, ideally scaling on a custom request-driven metric rather than the same CPU metric VPA is managing, to avoid the two autoscalers conflicting.
Are spot instances safe for production Kubernetes workloads?
Yes, for stateless, horizontally replicated workloads with PodDisruptionBudgets configured and pods spread across multiple instance types and availability zones. Stateful, single-replica services, or anything intolerant of a sudden two-minute termination notice, should stay on standard on-demand or reserved nodes instead.
Does the Kubernetes cluster autoscaler already handle bin-packing?
Not fully. The standard cluster autoscaler mainly adds nodes to accommodate pending pods and removes nodes once they are completely empty. Dedicated consolidation tools like Karpenter go further, actively repacking already-running workloads onto fewer nodes to reclaim capacity stranded across a fragmented fleet.
How much can right-sizing alone realistically save?
Savings vary by workload, but given that Cast AI’s data shows average CPU utilization near 8 to 10 percent of what is provisioned, closing even half that gap through accurate requests typically produces the single largest cost reduction available before touching discount programs like spot capacity at all.
Why do Kubernetes costs need separate tooling from regular cloud cost dashboards?
Standard cloud billing shows total node spend, not how that spend breaks down by namespace, team, or workload inside a shared cluster. Kubernetes cost tools attribute usage at the pod level, which is necessary for chargeback, prioritizing right-sizing work, and holding individual teams accountable for their actual resource consumption.
What is the risk of setting resource limits much higher than requests?
The Kubernetes scheduler places pods based primarily on requests, so a large gap between requests and limits does not affect scheduling directly, but it does allow a single noisy pod to consume far more of a node’s actual capacity than planned, indirectly undermining bin-packing efficiency for every other pod sharing that node.
Is Kubernetes cost optimization a one-time project or an ongoing process?
It is ongoing. Workload resource needs drift with every code deploy and every traffic pattern change, so a single right-sizing pass degrades within weeks. Teams that sustain savings run VPA recommendations, bin-packing, and spot allocation as continuous, automated processes rather than periodic manual audits.
References
- Cast AI — 2026 State of Kubernetes Optimization Report
- CNCF — The CNCF Annual Cloud Native Survey: The Infrastructure of AI’s Future
- Kubernetes documentation — “Vertical Pod Autoscaling”
- AWS documentation — “Amazon EC2 Spot Instances”
- Google Cloud documentation — “Spot VMs”
- FinOps Foundation — The State of FinOps Report 2025
