#data

Kafka was never built to be queried. Apache Fluss bolts a columnar, Arrow-native hot store onto the streaming layer and tiers cold to Iceberg — a clean full-stack realtime design whose only real open question is governance.

"Catalog" means two different things in the lakehouse: the technical catalog in your query path (Unity, Polaris) and the governance catalog beside it (OpenMetadata, DataHub). The second is where lineage, ownership, and trust live — and where the next fight is.

Most companies' data is not big enough to justify a distributed warehouse. A single fat box running DuckDB reads Parquet and Iceberg off S3 directly and answers the median analytics query in under a second, for a fixed bill and no cold start. The big-data era was mostly oversizing.

Iceberg and Delta reimplemented a transactional catalog as JSON and Avro files in object storage — and then needed a real database catalog on top anyway. DuckLake's heresy is to skip the file layer entirely: put all the metadata in SQL, keep the data in Parquet. It is both obvious and a little rude.

Iceberg did not win on features. Delta Lake had the bigger installed base and Hudi had the better write path. Iceberg won on governance and an engine-neutral spec, and the moment Databricks paid roughly $2B for Tabular the war was effectively over.

"Zero-ETL" is the data industry's most successful rebrand of the year — and the T never went anywhere. The transformation work does not vanish, it relocates to query time or to the catalog. What is genuinely new is the collapse in data duplication, and that part is worth the hype.

The .yml extension is a 1990s DOS artifact. The 'YAML Ain't Markup Language' acronym is a 2002 self-correction. Both questions resolve cleanly once you know markup languages and data serialisation formats are different categories with different ancestors.

A serverless warehouse that auto-provisions three nodes for a SELECT * spends most of the bill on the time you waited for it. A self-hosted ClickHouse on EC2 trades elasticity for sub-second latency and a fixed monthly line item — favourable any time queries are even mildly steady.

A lakehouse on object storage looks cheap because storage is cheap. The bill is built from request count and managed-tier access fees, both of which scale with file count, not data volume. 5 GB stored as one million 5 MB files is a different invoice than 5 GB stored as ten 512 MB files.

Douglas Crockford has said for twenty years that he did not invent JSON, he discovered it. The format was sitting inside JavaScript the whole time, waiting for someone to extract it. The story of how a 2001 footnote in a browser scripting language ate XML's lunch is shorter than most people think.