#iceberg
3 posts
· min read
DuckLake: metadata belongs in a database, not a pile of files
Iceberg and Delta reimplemented a transactional catalog as JSON and Avro files in object storage — and then needed a real database catalog on top anyway. DuckLake's heresy is to skip the file layer entirely: put all the metadata in SQL, keep the data in Parquet. It is both obvious and a little rude.
#data
#ducklake
#duckdb
#iceberg
#lakehouse
#opinion
#ai-assisted
· min read
How Apache Iceberg won the table-format war
Iceberg did not win on features. Delta Lake had the bigger installed base and Hudi had the better write path. Iceberg won on governance and an engine-neutral spec, and the moment Databricks paid roughly $2B for Tabular the war was effectively over.
#data
#lakehouse
#iceberg
#databases
#infrastructure
#opinion
#ai-assisted
· min read
The hidden cost of a lakehouse on S3
A lakehouse on object storage looks cheap because storage is cheap. The bill is built from request count and managed-tier access fees, both of which scale with file count, not data volume. 5 GB stored as one million 5 MB files is a different invoice than 5 GB stored as ten 512 MB files.
#data
#lakehouse
#s3
#iceberg
#cost
#ai-assisted