Amazon S3 Tables vs Self-Managed Apache Iceberg on S3: A Technical Deep-Dive for Startups
To manage or not to manage, that is the question.... This article provides Startups with a comprehensive guide as to whether to choose AWS-managed Iceberg on S3 tables or whether to self-manage Iceberg on general purpose S3 buckets for their analytics workloads.
Introduction
Apache Iceberg has become the de facto open table format for building data lakes on Amazon S3. For startups, the question is no longer whether to use Iceberg, but how to run it. AWS offers two paths: Amazon S3 Tables, a fully managed Iceberg experience built into S3 itself, and the self-managed approach using Iceberg on general purpose S3 buckets with the AWS Glue Data Catalog. Both deliver the core Iceberg capabilities (ACID transactions, schema evolution, time travel, partition evolution), but they differ in architecture, operational overhead, performance, and cost.
This guide breaks down the trade-offs to help technical decision-makers at startups pick the right path for their data lake.
Feature Comparison Summary
Category | Feature | S3 Tables (Managed) | Self-Managed Iceberg on General Purpose S3 |
|---|---|---|---|
Architecture | Storage Type | Purpose-built table buckets | Standard general purpose S3 buckets |
| Table Format | Apache Iceberg (built-in) | Apache Iceberg (you configure) |
| Catalog | S3 Tables REST Catalog / AWS Glue Data Catalog integration | AWS Glue Data Catalog |
| Bucket Type | Table bucket (new S3 bucket type) | General purpose bucket |
| Max Storage | S3 scale (up to 10,000 tables per bucket) | Unlimited objects per bucket |
Maintenance | Compaction | Automatic (binpack, sort, z-order) | AWS Glue Data Catalog optimizers or custom jobs |
| Snapshot Management | Automatic expiration and cleanup | Glue optimizer or manual |
| Unreferenced File Removal | Automatic | Glue orphan file deletion or manual |
| Record Expiration | Configurable, automatic | Manual implementation required |
Performance | Query Throughput | Up to 3x faster than self-managed | Baseline PostgreSQL-style Iceberg performance |
| Transactions Per Second | Up to 10x higher TPS | Standard Iceberg TPS |
| Compaction Strategies | Binpack, sort, z-order (automatic) | Binpack, sort, z-order (via Glue optimizers) |
Cost Model | Storage | Per GB-month (S3 Tables Standard pricing) | Per GB-month (S3 Standard or other storage classes) |
| Requests | Per 1,000 requests (PUT/GET) | Per 1,000 requests (standard S3 pricing) |
| Object Monitoring | $0.025 per 1,000 objects/month | None |
| Compaction | Per object + per GB processed (S3-managed) | Compute charges (Glue DPU-hours or EMR) |
| Table Updates | Per 1,000 table updates (for replication) | N/A |
Integration | Query Engines | Athena, Redshift, EMR, Glue, any Iceberg-compatible engine | Athena, Redshift, EMR, Glue, and any Iceberg-compatible engine |
| Governance | Lake Formation, IAM table-level policies | Lake Formation, IAM bucket/prefix policies |
| Replication | Built-in cross-region/cross-account replication | S3 Replication Rules (object-level, not table-aware) |
| Storage Classes | S3 Standard, Intelligent-Tiering | All S3 storage classes |
Availability | Regions | 32 AWS Regions | All AWS Regions |
| Iceberg Version | V2 and V3 | V2 and V3 |
Architecture
S3 Tables: Purpose-Built Table Storage
Amazon S3 Tables, launched at re:Invent 2024, introduces a new S3 bucket type called a table bucket, designed specifically for storing tabular data in Apache Iceberg format. The key architectural characteristics:
- Compute/Storage Separation: S3 Tables separates the storage layer from compute engines entirely. Any Iceberg-compatible engine (Athena, Redshift, EMR, Spark, Flink) can read and write to the same tables.
- Built-in Iceberg REST Catalog: S3 Tables exposes an Iceberg REST Catalog endpoint, meaning any Iceberg client can connect directly without custom catalog configuration.
- Automatic Maintenance: S3 manages compaction, snapshot expiration, and unreferenced file removal as background operations. No compute provisioning required on your side.
- Table-Level Access Control: IAM policies and resource-based policies can be attached directly to individual tables and table buckets, providing granular security without relying on S3 prefix conventions.
- SageMaker Lakehouse Integration: Table buckets integrate with Amazon SageMaker Lakehouse, populating the AWS Glue Data Catalog automatically and enabling unified governance through Lake Formation.
Self-Managed Iceberg on General Purpose S3
The self-managed approach stores Iceberg tables as files in standard S3 general purpose buckets, with metadata tracked in the AWS Glue Data Catalog:
- Standard S3 Storage: Data and metadata files live in general purpose buckets. You control the file layout, storage classes, and lifecycle policies.
- AWS Glue Data Catalog as Metastore: The Glue Data Catalog serves as the Iceberg catalog, tracking table schemas, partitions, and snapshots. Crawlers or direct registration populate the catalog.
- Glue Data Catalog Optimizers: AWS Glue offers managed table optimizers for compaction, snapshot retention, and orphan file deletion. These run as Glue jobs using DPU-hours, with compute costs billed separately.
- Full Storage Class Flexibility: You can use any S3 storage class (Standard, Standard-IA, Glacier Instant Retrieval, Intelligent-Tiering) and configure lifecycle rules at the object level.
- Broader Engine Compatibility: Since data sits in standard S3 buckets, any tool that reads Parquet/Avro/ORC from S3 can access the raw files, even without Iceberg awareness.
Maintenance: The Operational Divide
This is where the two approaches diverge most for startups. Table maintenance is the unglamorous but critical work that keeps Iceberg tables performant over time.
S3 Tables: Maintenance on Autopilot
S3 Tables handle four maintenance operations automatically:
- Compaction: Combines small files into larger ones to reduce metadata overhead and improve query scan efficiency. Supports three strategies:
- Binpack: Default. Combines files to reach a target size (configurable, default 512 MB).
- Sort: Organizes data by specified columns, reducing files scanned for filtered queries. Applied automatically when a sort order is defined in table metadata.
- Z-order: Interleaves values from multiple columns for queries filtering on several dimensions simultaneously. Enabled via the S3 Tables maintenance API.
- Snapshot Management: Automatically expires old snapshots based on configurable retention (default: minimum 1 snapshot, maximum 120 hours age). When snapshots expire, associated data files are marked noncurrent and cleaned up.
- Unreferenced File Removal: Deletes data and metadata files no longer referenced by any snapshot. Default retention of 3 days for unreferenced files and 10 days for noncurrent files.
- Record Expiration: Configurable automatic removal of expired records based on a timestamp column, available with S3 Tables Intelligent-Tiering.
All of this runs without you provisioning any compute. S3 handles scheduling, execution, and conflict resolution with Iceberg's optimistic concurrency model.
Self-Managed: You Own the Maintenance Pipeline
With general purpose S3 buckets, you have two options for maintenance:
Option A: AWS Glue Data Catalog Optimizers (Semi-Managed)
The Glue Data Catalog offers managed optimizers for Iceberg tables:
- Compaction: Supports binpack, sort, and z-order strategies. Triggered when partitions exceed 100 files smaller than 75% of target size.
- Snapshot Retention: Schedule-based cleanup of old snapshots.
- Orphan File Deletion: Periodic identification and removal of unreferenced files.
These can be enabled at the catalog level (applying to all new tables) or per-table. The key difference: Glue optimizers run as compute jobs billed at Glue DPU-hour rates. You also need to configure an IAM role with appropriate permissions.
Option B: Custom Maintenance Jobs
You build and schedule your own maintenance using:
- Spark jobs on EMR or Glue ETL calling Iceberg's
rewriteDataFiles,expireSnapshots, anddeleteOrphanFilesprocedures
- Scheduled Lambda functions or Step Functions orchestrating maintenance workflows
- Open-source tools like Iceberg's built-in maintenance actions
This gives maximum control but requires engineering time to build, monitor, and maintain.
Maintenance Comparison
Maintenance Task | S3 Tables | Self-Managed (Glue Optimizers) | Self-Managed (Custom) |
|---|---|---|---|
Compaction | Automatic, no compute cost to you (included in S3 Tables pricing) | Glue DPU-hour compute charges | Your compute (EMR/Glue/Lambda) |
Snapshot Expiration | Automatic | Schedule-based, Glue compute | Your compute + scheduling |
Orphan File Cleanup | Automatic | Schedule-based, Glue compute | Your compute + scheduling |
Record Expiration | Configurable | Not available (build custom) | Your compute + logic |
Sort/Z-order Compaction | Built-in | Available via Glue optimizers | Your Spark jobs |
Conflict Resolution | Handled by S3 | Handled by Glue optimizer | You implement retry logic |
Operational Overhead | Near zero | Low-moderate | High |
Performance
S3 Tables delivers measurably better performance for analytics workloads:
- Up to 3x faster query throughput compared to self-managed Iceberg tables on general purpose S3 buckets. This comes from the purpose-built storage layer optimized for tabular access patterns.
- Up to 10x higher transactions per second, critical for workloads with frequent concurrent writes from multiple pipelines.
- Automatic compaction keeps tables in an optimized state continuously, preventing the gradual performance degradation that occurs when small files accumulate in self-managed setups.
For self-managed tables, query performance depends heavily on how well you maintain your tables. A well-compacted, properly partitioned self-managed table can perform comparably to S3 Tables for read-heavy workloads. But maintaining that state requires consistent operational discipline.
The performance gap widens as data volume and write frequency increase. Startups with high-velocity data ingestion (streaming, frequent ETL updates) will see the most benefit from S3 Tables' automatic optimization.
High Availability and Disaster Recovery
Cross-Region Replication
S3 Tables: Built-in replication support creates read-only replica tables in destination table buckets across regions or accounts. Replication preserves parent-child snapshot relationships, and replica tables are queryable using any Iceberg-compatible engine. Updates are typically replicated within minutes. Supports both Iceberg V2 and V3 formats.
Self-Managed: S3 Cross-Region Replication (CRR) operates at the object level. It replicates the Parquet data files and Iceberg metadata files, but it is not table-aware. You need to ensure metadata consistency yourself, which can be complex when dealing with concurrent writes and compaction. Alternatively, you can build custom replication pipelines using Spark or Glue jobs.
Backup and Recovery
Both approaches benefit from S3's 99.999999999% (11 nines) durability. Iceberg's snapshot-based architecture provides built-in time travel for both, allowing you to query or roll back to previous table states.
S3 Tables automatically manage snapshot retention, ensuring you always have recovery points within your configured window. Self-managed tables require you to configure and monitor snapshot retention yourself.
Scalability
Storage Scaling
S3 Tables: Storage grows automatically in 10 GiB increments. No capacity planning required. Default quota of 10 table buckets per region (adjustable) with up to 10,000 tables per bucket and 10,000 namespaces per bucket.
Self-Managed: Standard S3 scaling applies. No practical limits on storage. Up to 10,000 general purpose buckets per account. No table-level quotas since tables are just file collections.
Compute Scaling
S3 Tables: Maintenance compute is fully managed by S3. Query compute scales with your chosen engine (Athena serverless, Redshift Serverless, EMR auto-scaling).
Self-Managed: Maintenance compute must be provisioned and scaled by you. Glue optimizers handle some of this, but you pay for DPU-hours. Query compute is the same as S3 Tables since both use the same engines.
Cost Analysis
S3 Tables Pricing Components
- Storage: $0.0265/GB-month (S3 Tables Standard, first 50 TB, US regions). Intelligent-Tiering available with automatic tiering to Infrequent Access (40% cheaper after 30 days) and Archive Instant Access (68% cheaper after 90 days).
- Requests: $0.005 per 1,000 PUT requests, $0.0004 per 1,000 GET requests.
- Object Monitoring: $0.025 per 1,000 objects/month. This is a fee specific to S3 Tables that does not exist for general purpose buckets.
- Compaction: $0.002 per 1,000 objects processed + $0.005/GB processed (binpack). Sort and z-order compaction have higher per-byte fees but were reduced by up to 90% in July 2025.
- Replication (if used): PUT request charges + $0.010 per 1,000 table updates + inter-region data transfer.
Self-Managed Pricing Components
- Storage: $0.023/GB-month (S3 Standard, first 50 TB, US regions). Full flexibility across all storage classes including S3 Standard-IA ($0.0125/GB), Glacier Instant Retrieval ($0.004/GB), and Intelligent-Tiering.
- Requests: $0.005 per 1,000 PUT requests, $0.0004 per 1,000 GET requests (same as general purpose S3).
- Glue Data Catalog: Free for the first million objects stored and first million requests/month. $1.00 per 100,000 objects stored above free tier.
- Glue Optimizer Compute: Billed at Glue DPU-hour rates when using managed compaction, snapshot retention, or orphan file deletion. Alternatively, EMR or custom compute costs for DIY maintenance.
- No Object Monitoring Fee: General purpose buckets do not have the per-object monitoring charge that S3 Tables incur.
TCO Modeling Example
Scenario: Startup analytics platform with daily ETL
Requirements:
- 1 TB table, growing ~50 GB/month
- Daily ETL creating 1,000 new 5 MB files + 3 metadata files
- 500,000 GET requests/month from analysts and dashboards
- Compaction needed to maintain query performance
- Single region, no replication
- US West (Oregon)
S3 Tables (Standard):
Plain text
Storage: 1,024 GB × $0.0265/GB = $27.14PUT requests: 30,090 requests × $0.005/1,000 = $0.15GET requests: 500,000 requests × $0.0004/1,000 = $0.20Object monitoring: 10,486 objects × $0.025/1,000 = $0.26Compaction: 30,000 objects × $0.002/1,000 = $0.06146.48 GB processed × $0.005/GB = $0.73─────────────Total: $28.54/month
Self-Managed on General Purpose S3 (with Glue Optimizers):
Plain text
Storage: 1,024 GB × $0.023/GB = $23.55PUT requests: 30,090 requests × $0.005/1,000 = $0.15GET requests: 500,000 requests × $0.0004/1,000 = $0.20Glue Catalog: Free tier (under 1M objects) = $0.00Glue Compaction: ~2 DPU × 0.5 hours × 30 runs× $0.44/DPU-hour = $13.20─────────────Total: $37.10/month
Cost Difference: S3 Tables saves ~$8.56/month (23%) in this scenario, primarily because S3 Tables compaction pricing is significantly cheaper than provisioning Glue compute for the same work.
Note: The Glue compaction estimate assumes modest DPU usage. Actual costs vary based on data volume, file count, and compaction frequency. For larger tables or more frequent compaction, the Glue compute costs scale up while S3 Tables compaction costs remain proportional to data processed.
When S3 Tables Wins on Cost
- High write frequency: More files created means more compaction needed. S3 Tables' per-object/per-GB compaction pricing scales more predictably than Glue DPU-hours.
- Multiple tables: The operational cost of managing maintenance across dozens of tables adds up quickly with the self-managed approach.
- Growing data: As tables grow, S3 Tables' automatic maintenance avoids the need to right-size Glue job configurations.
When Self-Managed Wins on Cost
- Cold or archival data: General purpose buckets offer access to cheaper storage classes (Standard-IA, Glacier Instant Retrieval) that S3 Tables does not yet support beyond Intelligent-Tiering.
- Minimal maintenance needs: Small, append-only tables that rarely need compaction avoid the object monitoring fee without needing Glue compute.
- Existing infrastructure: If you already run EMR clusters or Glue jobs for other purposes, adding Iceberg maintenance to existing compute may have near-zero marginal cost.
Query Engine Compatibility
Both approaches integrate with the same set of AWS analytics services:
Engine | S3 Tables | Self-Managed |
|---|---|---|
Amazon Athena | Yes | Yes |
Amazon Redshift / Redshift Serverless | Yes | Yes |
Amazon EMR (Spark, Flink, Presto) | Yes | Yes |
AWS Glue ETL | Yes | Yes |
Amazon SageMaker Unified Studio | Yes | Yes |
Amazon Data Firehose | Yes | Yes |
Quick Suite | Yes | Yes |
Apache Spark (open-source) | Yes (via REST Catalog) | Yes (via Glue Catalog) |
PyIceberg | Yes (via REST Catalog) | Yes (via Glue Catalog) |
Dremio, Snowflake, Starburst | Yes (partner integrations) | Yes (via Glue Catalog or direct S3) |
DuckDB | Yes (partner integration) | Yes (direct S3 read) |
The practical difference: S3 Tables expose an Iceberg REST Catalog endpoint, which is the emerging standard for Iceberg catalog interoperability. Self-managed tables rely on the Glue Catalog, which requires AWS-specific configuration in each engine. For multi-engine environments, the REST Catalog approach can simplify setup.
Governance and Security
S3 Tables
- Table-level IAM policies: Attach resource-based policies to individual tables and table buckets.
- Lake Formation integration: Fine-grained column-level and row-level access control through SageMaker Lakehouse.
- Encryption: SSE-S3 (default) or SSE-KMS with customer-managed keys.
- VPC connectivity: Supported via PrivateLink.
- Limitations: No public access policies, no presigned URLs, no HTTP requests (HTTPS only), IPv6 not supported for table/bucket-level actions.
Self-Managed
- Bucket and prefix-level IAM policies: Standard S3 bucket policies and IAM policies. Table-level granularity requires careful prefix-based policy design.
- Lake Formation integration: Full support for Glue Data Catalog tables, including column-level and row-level security.
- Encryption: All S3 encryption options (SSE-S3, SSE-KMS, SSE-C, client-side).
- Full S3 feature set: Presigned URLs, public access (if needed), all access patterns supported.
For startups with strict data governance requirements, both approaches support Lake Formation. S3 Tables simplify the security model with native table-level policies, while self-managed tables offer more flexibility at the cost of more complex policy management.
Iceberg Feature Support
Both approaches support Apache Iceberg V2 and V3, including:
- ACID transactions and serializable isolation
- Schema evolution (add, drop, rename, reorder columns)
- Partition evolution (change partitioning without rewriting data)
- Hidden partitioning
- Time travel and snapshot queries
- Row-level deletes and updates
- Deletion vectors (V3)
- Row lineage (V3)
The key difference is in how you interact with these features:
S3 Tables: Table creation, schema definition, and maintenance configuration happen through S3 APIs, the S3 console, or SQL via connected engines. The Iceberg REST Catalog handles metadata operations.
Self-Managed: You use Spark, Athena, or other engines to create and manage tables directly. The Glue Data Catalog stores metadata. You have full control over Iceberg table properties and configurations.
Limitations and Constraints
S3 Tables Limitations
- Regional availability: Available in 32 AWS Regions as of early 2026, but not all regions. Check current availability for your target regions.
- Storage classes: S3 Standard and Intelligent-Tiering only. No Standard-IA, One Zone-IA, or Glacier classes.
- Table bucket quotas: Default 10 table buckets per region (adjustable), 10,000 tables per bucket, 10,000 namespaces per bucket.
- No presigned URLs: Cannot generate presigned URLs for direct object access.
- No public access: Table buckets do not support public access policies.
- File format: Apache Iceberg only. Cannot store non-Iceberg data in table buckets.
- Lowercase only: Table names and definitions must use all lowercase letters for compatibility with AWS analytics services.
Self-Managed Limitations
- Maintenance burden: You are responsible for compaction, snapshot management, and orphan file cleanup. Neglecting maintenance degrades query performance over time.
- Catalog management: Glue Data Catalog requires configuration and monitoring. Schema drift, stale metadata, and catalog inconsistencies are your problem.
- No built-in replication: S3 CRR is object-level, not table-aware. Maintaining consistent replicas across regions requires custom engineering.
- Performance ceiling: Without the purpose-built storage optimizations of S3 Tables, you may hit performance limits under high-concurrency workloads.
- Glue optimizer limitations: Compaction only triggers when partitions exceed 100 files smaller than 75% of target size. Less granular control than S3 Tables' continuous optimization.
Migration Path
AWS provides a migration path from self-managed Iceberg tables on general purpose S3 buckets to S3 Tables. The process involves:
- Creating a table bucket and target table in S3 Tables
- Using Spark (on EMR, Glue, or SageMaker) to read from the source Glue Catalog table and write to the S3 Tables destination
- Validating data integrity and updating downstream consumers to point to the new table
This is a data copy operation, not an in-place migration. Plan for the compute and time required to move your data.
Decision Framework
When to Choose S3 Tables
- You want zero-ops table maintenance: Compaction, snapshot management, and file cleanup happen automatically. Your team focuses on building pipelines and analytics, not maintaining infrastructure.
- High write throughput: Streaming ingestion, frequent ETL updates, or multiple concurrent writers benefit from 10x higher TPS and automatic compaction.
- Growing number of tables: Managing maintenance across 10, 50, or 100+ tables is trivial with S3 Tables. With self-managed, it becomes a significant operational burden.
- Query performance matters: The 3x query throughput improvement is meaningful for interactive dashboards and ad-hoc analytics.
- Cross-region replication needed: Built-in table-aware replication is simpler and more reliable than object-level S3 CRR for Iceberg tables.
- Small team, limited data engineering capacity: Startups with 1-3 data engineers benefit most from eliminating maintenance overhead.
When to Choose Self-Managed Iceberg
- Cost-sensitive archival workloads: If most of your data is cold and you need Glacier Instant Retrieval or Standard-IA pricing, general purpose buckets offer more storage class options.
- Existing mature data platform: If you already have well-tuned Glue jobs, EMR clusters, and maintenance pipelines, the marginal benefit of S3 Tables may not justify migration effort.
- Specific regional requirements: If your target region does not yet support S3 Tables, self-managed is your only option.
- Maximum flexibility: Full control over file layout, storage classes, lifecycle policies, and Iceberg table properties. No constraints from the managed service.
- Non-Iceberg data co-location: If you need to store Iceberg tables alongside non-tabular data (images, logs, ML artifacts) in the same bucket, general purpose buckets are required.
- Multi-cloud or hybrid requirements: If you need to access the same data from non-AWS environments using standard S3 APIs and presigned URLs, self-managed tables offer more access patterns.
Decision Tree
Plain text
Apache Iceberg Table Storage Selection│├─ Do you need storage classes beyond Standard and Intelligent-Tiering?│ ││ ├─ Yes → Self-Managed Iceberg on General Purpose S3│ ││ └─ No│ ││ └─ Is your target region supported by S3 Tables?│ ││ ├─ No → Self-Managed Iceberg on General Purpose S3│ ││ └─ Yes│ ││ └─ Do you have an existing, well-maintained Iceberg platform?│ ││ ├─ Yes│ │ ││ │ └─ Is maintenance overhead a pain point?│ │ ││ │ ├─ Yes → Migrate to S3 Tables│ │ ││ │ └─ No → Stay with Self-Managed (evaluate S3 Tables for new tables)│ ││ └─ No (starting fresh)│ ││ └─ Do you have dedicated data engineering capacity for maintenance?│ ││ ├─ Yes│ │ ││ │ └─ Is cost the primary constraint?│ │ ││ │ ├─ Yes → Self-Managed (optimize storage classes)│ │ ││ │ └─ No → S3 Tables│ ││ └─ No → S3 Tables
Hybrid Approach
Many startups will benefit from using both:
- S3 Tables for active, frequently queried tables: Production analytics, dashboards, real-time reporting.
- Self-Managed for archival and cost-optimized storage: Historical data, compliance archives, infrequently accessed datasets using cheaper storage classes.
- S3 Tables for new projects: Start new data products on S3 Tables to avoid building maintenance infrastructure.
- Self-Managed for existing workloads: Continue running stable, well-maintained existing tables until migration makes sense.
Best Practices
For S3 Tables
- Define sort orders on tables with predictable query patterns to enable automatic sort compaction.
- Enable Intelligent-Tiering for tables with mixed access patterns to automatically reduce storage costs on cold data.
- Use SageMaker Lakehouse integration for centralized governance across multiple table buckets and query engines.
- Monitor compaction metrics through CloudWatch to understand maintenance patterns and costs.
- Use namespaces to logically organize tables within a table bucket (e.g., by domain, team, or environment).
- Use lowercase for all table names and definitions to ensure compatibility across all AWS analytics services.
For Self-Managed Iceberg
- Enable Glue Data Catalog optimizers at the catalog level to automatically apply compaction, snapshot retention, and orphan file deletion to all new tables.
- Set appropriate target file sizes (default 512 MB) based on your query patterns and data volume.
- Monitor Glue optimizer runs through CloudWatch metrics and set alerts for failures.
- Implement partition strategies that align with your most common query filters.
- Schedule regular orphan file cleanup to prevent storage cost creep from abandoned files.
- Use Iceberg's
write.target-file-size-bytestable property to control file sizes at write time, reducing the need for post-write compaction.
General Best Practices
- Start with Iceberg V3 for new tables to take advantage of deletion vectors and row lineage.
- Use hidden partitioning to decouple physical data layout from query syntax.
- Implement schema evolution carefully, testing downstream consumers before making breaking changes.
- Enable column statistics for better query planning and scan optimization.
- Use time travel for debugging and auditing rather than maintaining separate audit tables.
Conclusion
For most startups building new data lakes on AWS, S3 Tables is the right default choice. The automatic maintenance, higher performance, and lower operational overhead let small teams focus on deriving value from data rather than managing infrastructure. The cost model is competitive, and often cheaper than self-managed when you factor in compute costs for maintenance.
Self-managed Iceberg on general purpose S3 remains the right choice when you need maximum storage class flexibility, operate in regions where S3 Tables are not yet available, or have an existing well-tuned platform that does not justify migration.
The good news: both approaches use Apache Iceberg, an open standard. Your data is not locked in. You can start with one approach and migrate to the other as your needs evolve.
References
- Amazon S3 Tables Documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables.html
- Amazon S3 Tables Product Page: https://aws.amazon.com/s3/features/tables/
- S3 Tables Pricing: https://aws.amazon.com/s3/pricing/
- AWS Glue Data Catalog Iceberg Optimizers: https://docs.aws.amazon.com/glue/latest/dg/table-optimizers.html
- Apache Iceberg on AWS Prescriptive Guidance: https://docs.aws.amazon.com/prescriptive-guidance/latest/apache-iceberg-on-aws/data-lakes.html
- S3 Tables Regional Availability: https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-regions-quotas.html
- S3 Tables Maintenance Documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-maintenance-overview.html
- S3 Tables Replication and Intelligent-Tiering: https://aws.amazon.com/blogs/aws/announcing-replication-support-and-intelligent-tiering-for-amazon-s3-tables/
- S3 Tables Compaction Cost Reduction (July 2025): https://aws.amazon.com/about-aws/whats-new/2025/07/amazon-s3-tables-reduce-compaction-costs/
- Apache Iceberg V3 on AWS: https://docs.aws.amazon.com/AmazonS3/latest/userguide/working-with-apache-iceberg-v3.html
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article