Uploaded February 2023 | Updated September 2026, 9 hours ago
Log-Structured Merge trees (LSM trees) are increasingly used as the storage engines behind several data systems, frequently deployed in the cloud. Similar to other database architectures, LSM trees consider information about the expected workload (e.g., reads vs. writes, point vs. range queries) to optimize their performance via tuning. However, operating in a shared infrastructure like the cloud comes with workload uncertainty due to the fast-evolving nature of modern applications. Systems with static tuning discount the variability of such hybrid workloads and hence provide inconsistent and overall suboptimal performance. In this talk we introduce Endure – a new paradigm for tuning LSM trees in the presence of workload uncertainty. We introduce the robust problem and show how it differs from the traditional tuning problem. Then we will discuss how to define uncertainty and use this definition to recommend tunings that perform on average better under the presence of a changing workload. We show the potential of robust tunings by implementing Endure in RockDB, a popular LSM engine, and analyze the performance benefits of using robust tunings.
Speakers:
Andy Huynh, Boston University
Manos Athanassoulis, Boston University
Evimaria Terzi, Boston University
Conversation Leader:
Josh Berkus, Red Hat
Log-Structured Merge trees (LSM trees) are increasingly used as the storage engines behind several data systems, frequently deployed in the cloud. Similar to other database architectures, LSM trees consider information about the expected workload (e.g., reads vs. writes, point vs. range queries) to optimize their performance via tuning. However, operating in a shared infrastructure like the cloud comes with workload uncertainty due to the fast-evolving nature of modern applications. Systems with static tuning discount the variability of such hybrid workloads and hence provide inconsistent and overall suboptimal performance. In this talk we introduce Endure – a new paradigm for tuning LSM trees in the presence of workload uncertainty. We introduce the robust problem and show how it differs from the traditional tuning problem. Then we will discuss how to define uncertainty and use this definition to recommend tunings that perform on average better under the presence of a changing workload. We show the potential of robust tunings by implementing Endure in RockDB, a popular LSM engine, and analyze the performance benefits of using robust tunings.
Speakers:
Andy Huynh, Boston University
Manos Athanassoulis, Boston University
Evimaria Terzi, Boston University
Conversation Leader:
Josh Berkus, Red Hat
