How to get GPU Capacity on AWS
A practical, comprehensive guide to help you secure GPU capacity on AWS. Compare purchasing options and apply a decision framework to optimize cost, availability, and business value, supported by actionable best practices.
GPU capacity on AWS powers a wide range of AI workloads, from startups fine-tuning foundation models to enterprises deploying production-scale inference endpoints. AWS provides flexible capacity options designed to help teams optimize both cost and availability for their specific workload requirements.
I've worked with dozens of customers successfully leveraging these capabilities. For example, an engineering team in AI research startup implemented fault-tolerant queuing in their production inference pipeline, enabling them to utilize Spot instances for the majority of their workload. By combining Spot with reserved capacity for critical phases, they reduced infrastructure costs by approximately 60% compared to their previous approach. Another machine learning team pre-planned training and used EC2 Capacity Blocks to secure GPU capacity for critical releases, reducing uncertainty and avoiding budget overruns.
This article explores practical strategies to help you secure GPU capacity on AWS while optimizing for cost efficiency and business value. It covers the main purchasing options, compares their cost–benefit tradeoffs with real pricing examples, and provides a decision framework to help teams match workload requirements to the right capacity model, along with actionable best practices for maximizing availability through regional flexibility, quota management, and hybrid purchasing strategies.
Match the Purchase Option to Your Workload Requirements
AWS offers multiple purchasing options designed to match your specific workload requirements. Whether you need cost optimization, reserved capacity, or maximum flexibility, you can choose from On-Demand, Spot, Capacity Blocks, and On-Demand Capacity Reservations (ODCRs), each providing distinct benefits around cost, flexibility, and capacity assurance to meet your business needs.
Your decision should be driven by key factors including strategic importance, commitment readiness, time sensitivity, demand predictability, instance and regional flexibility, duration, cost priorities, and interruption tolerance.
Demand predictability and flexibility across instance types, regions, and timing unlock more options and better pricing. In practice, teams adopt a hybrid approach, combining purchasing options to balance cost, availability, and reliability for their specific workloads.
GPU Purchase Decision Tree
A prerequisite step is to verify that the relevant instances are available in your target region. Instance and region flexibility will improve your chances of securing capacity. Choosing the right instance type for your workload can significantly improve cost-performance. Assess your performance needs and cost constraints before deciding.
Purchase Option Comparison
Pricing Example
The following table provides an example of pricing differences across purchase options. These point-in-time prices are examples from N. Virginia (us-east-1) as of March 2026 and are subject to change based on supply and demand dynamics. Regional pricing varies significantly. Always verify current pricing.
- Amazon EC2 On-Demand Pricing
- Compute and EC2 Instance Savings Plans
- Amazon EC2 Capacity Blocks for ML pricing
- Amazon EC2 Spot Instances Pricing
- AWS Pricing Calculator
- Your AWS account team for custom pricing discussions
- Effective Hourly Rate per GPU ($/hour)
- For example, a p5.48xlarge Spot instance can reduce costs by approximately 70% compared to On-Demand pricing.
Capacity Purchase Options
Each purchase option has distinct tradeoffs around cost, flexibility, and capacity assurance.
Spot Instances
Amazon EC2 Spot Instances are EC2 capacity that AWS makes available at a discounted price, up to 90% compared to On-Demand, because it can be interrupted when required, with a 2-minute interruption notice. This makes Spot highly attractive for cost-sensitive, fault-tolerant workloads. Importantly, Spot capacity operates independently from On-Demand capacity, Spot Instances may be available even when On-Demand capacity is constrained.
For P-series GPU instances, EC2 sends Rebalance Recommendations on a best-effort basis up to 30 minutes before interruption, compared to the standard 2-minute interruption notice. Spot Placement Score provides per-instance visibility into capacity availability for P-series GPU instances. Spot instances integrate well with tools like Amazon EKS and Karpenter.
While they offer significant cost savings, workloads must be interruption-tolerant. A common workload includes fault-tolerant training with checkpointing, large-scale hyperparameter sweeps, batch inference jobs, and real-time region-flexible inference.
EC2 Capacity Blocks for ML
Capacity Blocks for ML allow you to reserve GPU-based accelerated computing instances on a future date to support your short duration machine learning (ML) workloads. Capacity Blocks currently support P-series GPU instances with booking windows up to eight weeks in advance, subject to availability, for durations of up to six months, in cluster sizes from 1 to 64 instances. This model is ideal for planned workloads that require high certainty of capacity, but it requires advance planning. You pay upfront to reserve capacity, and reservations are non-refundable. Designed for defined, short-duration training and fine-tuning workloads, including planned large-scale training runs, and time-bound research experiments.
On-Demand Instances
On-Demand Instances let you pay for compute capacity by the hour or second with no long-term commitments. You can launch or terminate instances anytime without long-term obligations. This makes On-Demand a simple starting point for GPU workloads, suitable for development, prototyping, and unpredictable or short-lived workloads. On-Demand instances do not provide capacity guarantees. If you need higher assurance of capacity availability, it is recommended to use a reservation. You can reduce costs by combining On-Demand with Savings Plans, which offer up to 72% savings with a 1- or 3-year commitment.
On-Demand Capacity Reservations (ODCRs)
On-Demand Capacity Reservations (ODCR) allow you to reserve GPU capacity on AWS without long-term commitments, reserving availability in a specific Availability Zone for any duration you need. ODCRs are billed at standard On-Demand rates, whether the reserved instances are actively used.
You can create immediate reservations (subject to current capacity) or future-dated reservations scheduled 5–120 days in advance, with AWS attempts to fulfill future-dated reservations based on projected capacity, though availability is not guaranteed until the reservation is confirmed. Once confirmed, your capacity is reserved for the specified window. Future-dated ODCRs may have minimum duration requirements that vary by instance type and region.
While ODCRs are designed for capacity assurance rather than cost savings, Savings Plans discounts can still apply to matching usage. It’s also important to note that future-dated reservations are currently supported for G-Series instances.
Suited for longer training and inference workloads with a defined baseline, including production inference services, scheduled training jobs with specific timing requirements, and business-critical applications.
Service Quota Management
Before launching GPU instances, ensure your account has sufficient vCPU quota for the relevant instance family. AWS enforces separate quotas per family and purchase model, including:
- Running On-Demand P instances
- Running On-Demand G and VT instances
- All P Spot Instance Requests
- All G and VT Spot Instance Requests
Importantly, these quotas are based on vCPU count, not instance count. For example, launching 10 × p5.48xlarge instances requires 1,920 vCPUs (10 × 192). You can check your limits in the AWS Console under Service Quotas → EC2 → [Quota name], and request increases directly from the same interface.
If you encounter quota limitations when creating Capacity Blocks Reservations, launching On-Demand instances, or submitting Spot requests, contact AWS Support or your AWS account team.
Best Practices
Consider the following best practices to improve capacity availability:
- Reserve early – For critical workloads, secure capacity weeks in advance using Capacity Blocks or future-dated reservations.
- Use all Availability Zones – Capacity often varies between AZs within the same Region.
- Consider multi-Region deployments – Expanding beyond a single Region significantly increases your chances of finding capacity.
- Be flexible on instance types and sizes – Supporting multiple families (e.g., P5 and P4d) improves availability.
- Adopt flexible purchasing models – Combine long-term reservations, Capacity Blocks, On-Demand, and Spot based on workload needs.
- Ensure quotas are sufficient – Verify and increase service limits in advance to avoid deployment failures.
Teams that maximize flexibility across instance types, Regions, timing, and purchasing models achieve better availability and more cost-efficient outcomes.
Conclusion
Start by assessing your workload requirements. The decision tree above, along with information on the different purchasing options, can help you identify the best fit for your needs. AWS offers a range of purchasing models tailored to different workloads, some provide immediate availability, while others require advance planning and strategic procurement.
Need additional guidance? Contact your AWS account team or AWS Support to discuss your requirements and explore options for securing the GPU capacity that best fits your needs.
If you would like to learn more:
- Navigating the AWS Accelerated Computing Instance choices
- Guidance for Building a Spot Placement Score Tracker Dashboard on AWS
- Comparing On-Demand Capacity Reservations and Capacity Blocks for ML
Disclaimer: Pricing information in this article is provided for educational purposes only and should not be relied upon for purchasing decisions. Consult current AWS pricing pages for your specific region and configuration.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article