AWS Builder Center

Builder Center

Bedrock vs SageMaker vs EKS - LLM Inference Cost Guide

Decision framework with 2026 pricing, latency benchmarks, and cost crossover points for choosing between Bedrock, SageMaker, and self-hosted LLM on EKS.

AWS gives you three paths to production LLM inference. Choosing wrong at 1,000 requests/day means expensive rework at 100,000. This guide covers the architecture, costs, and operational trade-offs of each option so you can pick the right one from the start.

The three options at a glance


Bedrock
SageMaker
Self-Hosted (EKS)
Model
Pay-per-token, fully managed
Pay-per-hour, semi-managed
Pay-for-compute, full control
Best for
<50K req/day, frontier models
Fine-tuned models, dedicated capacity
>100K req/day, open-source models
Ops overhead
None
Medium (endpoint config, scaling)
High (GPU scheduling, Karpenter, monitoring)
Cost at 50K req/day
~£1,450/month
~£810/month
~£540/month

When to choose Bedrock

Bedrock is the right default when:
  • You want zero infrastructure management
  • Inference volume is under 50,000 requests/day
  • You need access to frontier models (Claude Sonnet 5, Opus 4.8, Llama 3) without managing GPUs
  • UK data residency is satisfied by eu-west-2 or EU Geographic routing

Pricing example (July 2026)

A customer support chatbot handling 10,000 conversations/day with average 500 input tokens and 200 output tokens, using Claude 3.5 Haiku in eu-west-2:
  • Input: 10,000 × 500 / 1,000 × $0.0008 = $4.00/day
  • Output: 10,000 × 200 / 1,000 × $0.004 = $8.00/day
  • Monthly total: $360 (£290)
That's remarkably cheap for a production AI system with no infrastructure to manage.

Key limitation

Default quotas throttle high-volume workloads. If you hit  ThrottlingException  (HTTP 429), you'll need to implement exponential backoff and request quota increases proactively. The newer models (Sonnet 5, Opus 4.8) require EU Geographic routing - your data stays within EU but may be processed in Frankfurt or Ireland rather than strictly London.

When SageMaker wins

SageMaker fills the gap when you need:
  • Fine-tuned model serving (custom containers)
  • Predictable latency with dedicated capacity
  • Control over quantisation (GPTQ, AWQ) and batch sizes
  • VPC isolation without shared infrastructure

Cost model

Running a fine-tuned Llama 3 8B on ml.g5.xlarge 24/7:
  • $1.41/h × 720 hours = ~$1,015/month
The crossover: if you're processing more than ~250,000 requests/month with 500-token prompts, SageMaker becomes cheaper than Bedrock per request.

Autoscaling pattern

import boto3

client = boto3.client("application-autoscaling")

client.register_scalable_target(
    ServiceNamespace="sagemaker",
    ResourceId="endpoint/llm-production/variant/AllTraffic",
    ScalableDimension="sagemaker:variant:DesiredInstanceCount",
    MinCapacity=1,
    MaxCapacity=8,
)

client.put_scaling_policy(
    PolicyName="llm-invocations-scaling",
    ServiceNamespace="sagemaker",
    ResourceId="endpoint/llm-production/variant/AllTraffic",
    ScalableDimension="sagemaker:variant:DesiredInstanceCount",
    PolicyType="TargetTrackingScaling",
    TargetTrackingScalingPolicyConfiguration={
        "TargetValue": 100.0,
        "PredefinedMetricSpecification": {
            "PredefinedMetricType": "SageMakerVariantInvocationsPerInstance"
        },
        "ScaleOutCooldown": 60,
        "ScaleInCooldown": 300,
    },
)
# amazon-bedrock# amazon-sagemaker# amazon-eks# generative-ai# finops
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article