DuckLake - The SQL-Powered Lakehouse Format for the Rest of Us by Prof. Hannes Mühleisen @tigerbeetledb
DuckLake - The SQL-Powered Lakehouse Format for the Rest of Us by Prof. Hannes Mühleisen  @tigerbeetledb
Uploaded August 2025 | Updated November 2026, 12 hours ago
Managing data on Object Stores has been a painful affair. Users had to choose between data swamp chaos or a maze of metadata files with catalog servers on top.

DuckLake is a new paradigm for managing data on object stores: First, it uses classical SQL data management systems to manage metadata. Second, actual data is stored in Parquet files on pretty arbitrary storage. Third, processing queries is done client-side, or anywhere really. DuckDB is the first system to integrate with DuckLake using an extension with the same name.

Conceptually, DuckLake enables central control over truth while de-centralizing compute and storage entirely. DuckLake turns data warehouse architecture upside down by departing from the integrated metadata/compute layer towards a fully disconnected operation with only centralized metadata. For the first time, DuckLake allows a “multi-player” experience with DuckDB, where computation stays fully local, but transactional control is centralized.

In our talk, we will describe the design rationale of the DuckLake format and its principles of simplicity, scalability, and speed. We will show the DuckDB implementation of DuckLake in action and discuss the implications for data architecture in general.

hannes.muehleisen.org
duckdb.org
https://ducklake.select/

Talk from Systems Distributed '25: systemsdistributed.com
DuckLake - The SQL-Powered Lakehouse Format for the Rest of Us by Prof. Hannes MühleisenToolchain HorizonsEpisode 083: Adaptive Routing TheoryEpisode 077: The Upgrade TrickThe Making of Systems DistributedEpisode 072: Workload&Auditor, Creating AccountsEpisode 90: Next, Peek, Next, PopEpisode 115: Disentangling Corruption And Torn WritesEp. 15: API CleanupEpisode 106: Refactoring CLI ParsingEpisode 067: Logical and Physical DSTEpisode 066: Tree of Losers For The Win!
TigerBeetle |

DuckLake - The SQL-Powered Lakehouse Format for the Rest of Us by Prof. Hannes Mühleisen

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER