← Blog··Updated 21 May 2026·7 min read

The hidden cost of a lakehouse on S3

A lakehouse on object storage looks cheap because storage is cheap. The bill is built from request count and managed-tier access fees, both of which scale with file count, not data volume. 5 GB stored as one million 5 MB files is a different invoice than 5 GB stored as ten 512 MB files.

AI-assisted postDrafted with help from Claude, edited and fact-checked by Mart. See transparency policy →
A cloud-shaped invoice

The cloud bill is the part of cloud computing the architecture diagrams leave out.

The headline confusion

The pitch for a lakehouse on object storage is that storage is cheap. S3 Standard at $0.023 per GB-month is the number that ends up on the back of the napkin during the architecture review, and the napkin is correct: a terabyte of Parquet sitting in a bucket is roughly $23 a month, and that is genuinely a bargain compared with what the same terabyte would cost on a managed warehouse. The trouble is that the napkin only describes the line on the bill labelled storage. The other lines — Requests-Tier1, Requests-Tier2, DataTransfer-Regional-Bytes, and whatever the managed-storage vendor decides to call their per-GB data-access fee — scale with how the data is laid out, not how much of it there is. A badly-laid-out 5 GB table can cost more to write than a well-laid-out 5 TB one, and most lakehouse cost surprises live in exactly that gap.

What a lakehouse on S3 actually is

A lakehouse on S3 in 2026 means three things stacked on top of each other: a pile of Parquet (or ORC) data files on raw object storage, a table-format layer that gives them transactions and schema evolution (Apache Iceberg, Delta Lake, or Apache Hudi), and a catalog that resolves table names to the current snapshot (AWS Glue, Polaris, Unity, or a REST catalog). Compute is rented separately — Athena, Trino, Spark on EMR, Snowflake-managed Iceberg, Databricks. The whole point of the architecture is that the storage and the compute are decoupled, which is the same architectural move that turned the cloud into a product category in 2006 when AWS first sold S3 by itself, without a server attached. The cost surprises are the consequence of the decoupling: every query is a network conversation with a bucket, and every conversation is metered.

Where the bill actually comes from

There are three categories of charge that compound:

Storage. S3 Standard is $0.023/GB-month for the first 50 TB. S3 Tables, the Iceberg-aware managed bucket type AWS launched in late 2024, is $0.0265/GB — roughly a 15 % premium that buys automated compaction and unreferenced-file cleanup. Glacier Deep Archive is $0.00099/GB. The full storage-class table is on the S3 pricing page and is the one part of the bill that almost everyone budgets for correctly.

Requests. S3 charges per API call. On S3 Standard, PUT, COPY, POST, and LIST cost $0.005 per thousand. GET, SELECT, and most other reads cost $0.0004 per thousand. Roughly twelve and a half times cheaper to read than to write. The other load-bearing detail is in the ListObjectsV2 documentation: one call returns at most 1,000 keys. Listing a prefix containing a million objects is a thousand round trips, not one, and every one of them is a billable PUT-class request.

Egress. Same-region traffic between S3 and an EC2 instance, an Athena workgroup, or an EMR cluster is free. Cross-region S3 traffic costs $0.02 per GB on a typical corridor. Internet egress is $0.09/GB after the first 100 GB free per month, with the rate sliding down to $0.05/GB above 150 TB (current AWS data-transfer rates). One terabyte of query results going out to the public internet is $90; one terabyte being scanned by a cross-region compute cluster is $20; one terabyte being scanned by a same-region cluster is $0. The geometry of the architecture decides which of those three numbers shows up.

The small-files worked example

Take 5 GB of data laid out in the worst plausible shape: 1,000,000 Parquet files of 5 MB each. The storage line is about $0.115 a month. Everything else scales with the million.

A full table scan needs one GET per file. One million GETs at $0.0004 per thousand is $0.40 per scan. Listing those files takes 1,000 LIST pages at $0.005 per thousand, so $0.005 per scan in LIST charges. The dollar figures are not the painful part — at most query engines the latency from a thousand sequential LIST round trips is what kills the query — but the dollars climb fast under any kind of scheduled-job pattern: a hundred dashboard refreshes a day is $40 a month in GETs to read $0.115 of storage.

The write side is worse. Landing 1,000,000 files costs 1,000,000 PUTs at $0.005 per thousand, or $5.00 in PUT charges to ingest the same 5 GB. The data costs more to write than it costs to store for the next three and a half years.

The compacted version of the same table — ten 512 MB files instead of a million 5 MB ones — needs ten GETs and one LIST per scan, and cost roughly five cents in total to land. Same five gigabytes, identical Iceberg query semantics, four orders of magnitude fewer requests on every read.

One SELECT * on the worst-shaped 5 GB table

A single full scan from a co-located query engine — Athena, Trino on EMR, Redshift Spectrum — totals up to:

Line item Cost per scan
LIST (1,000 paginated calls) $0.005
GET (1,000,000 objects) $0.40
Athena scan ($5/TB × 5 GB) $0.025
Same-region egress to engine free
Per-scan total (same region) ~$0.43
Cross-region engine, add egress +$0.10
Per-scan total (cross-region) ~$0.53

A hundred dashboard refreshes a day on this table is roughly $1,290 a month to read 5 GB that costs $0.115 a month to store. The data is roughly 11,000× cheaper to keep than it is to ask about.

The compacted version of the same table runs about $0.004 per scan in GETs — two orders of magnitude cheaper for the identical query result.

flowchart LR
  subgraph small["Small files — 1M × 5MB"]
    qe1[Query engine] -->|LIST × 1000| s3a[(S3 bucket)]
    qe1 -->|GET × 1,000,000| s3a
  end
  subgraph big["Compacted — 10 × 512MB"]
    qe2[Query engine] -->|LIST × 1| s3b[(S3 bucket)]
    qe2 -->|GET × 10| s3b
  end

The managed-tier surcharge

The above is just the raw-S3 bill. On top of it, every managed-lakehouse vendor adds a per-GB data-access fee for traffic that goes through their own optimised storage path. A representative shape, taken from a current managed-lakehouse pricing sheet:

Operation Price
File reads from high-performance storage $0.03 / GB
File reads directly from S3 bucket Free
File writes $0.06 / GB

The numbers move from vendor to vendor, but the structural pattern repeats across Databricks Unity Catalog managed storage, Snowflake-managed Iceberg, and several smaller catalogs: there is a free tier when the engine reads raw S3, and a metered tier when it reads from the vendor's accelerated cache. The vendor's marketing positions the accelerated tier as a performance feature; the bill positions it as $0.03 per GB on every byte read. A team running a 1 TB daily scan on the managed path is paying $30 a day, or about $900 a month, in data-access fees alone — on top of S3 storage, on top of S3 requests, on top of compute. That line item does not appear in any storage cost calculator.

S3 Tables specifically

S3 Tables, which AWS launched in late 2024, is the cleanest example of how the vendors price around the small-files problem. The premium over S3 Standard buys automated Iceberg maintenance: binpack compaction, sort compaction, z-order compaction, and unreferenced-file expiry, all running as a background service. Compaction has its own per-object and per-byte fees, and AWS cut those fees in July 2025 by 50 % on the per-object component and up to 90 % on the per-byte component, after sustained customer complaints that the original prices made S3 Tables more expensive than self-managed Iceberg for any non-trivial table. Onehouse has a detailed walk-through of the original surprise and a follow-up after the cuts. The relevant point for the architecture discussion is that AWS treats the cleanup as its own SKU. The small-files problem is the product the managed tier sells against.

The compaction calculus

The Iceberg community's recommended target file size is 256–512 MB. Anything below 128 MB starts to be expensive in metadata overhead and query-planning latency. Anything above 1 GB starts to hurt parallelism, because each file is one parallel read unit at the engine. Compaction itself is not free — rewriting a thousand 5 MB files into one 5 GB file is one PUT plus a thousand DELETEs plus a manifest rewrite — but the cost amortises rapidly under any read-heavy workload. A table that gets scanned a hundred times a day pays the compaction back in days. A table that gets scanned once a quarter pays the compaction without ever getting the read savings, and is a candidate for not compacting at all. The compaction question is, in cost terms, a question about read frequency.

The egress trap

The third category, network egress, is the one that quietly turns into the biggest line on the bill when an architecture drifts. Same-region S3 traffic to compute is free. Cross-region S3 traffic is $0.02 per GB, every time. A team that puts their Iceberg bucket in us-east-1 for historical reasons and a Trino cluster in us-west-2 for latency reasons pays $0.02 on every byte of every scan, every day, recurring. A 1 TB daily scan in that shape is $20 a day, or about $600 a month, in pure network — for an architecture decision that was never costed because each individual query looks cheap. Internet egress is worse: pushing a 1 TB query result out of S3 directly to an external consumer is $90 in egress alone, which is why most data-sharing architectures route through CloudFront, S3-to-CloudFront in-region being free, or through a same-region API layer that pages the result rather than streaming it.

A short close

The lakehouse story sold the cheap half of the bill. The other three categories — request pricing, managed-tier data access, and cross-region or internet egress — together usually dominate the storage line by an order of magnitude, and all three scale with how the data is laid out, not how much of it there is. Compaction, partitioning, and region locality are the three knobs that decide which bill the team gets at the end of the month. Object storage being cheap is true. Reading a million objects from object storage being cheap is not.

Read next