Uploaded November 2025 | Updated September 2026, 2 weeks ago
Ordered Bucket Index Shards in RGW - Eric Ivancich, IBM
RGW on RADOS uses hash-based sharded bucket indices to store metadata associated w/ objects. The key advantage is that mapping an object name to a shard is constant-time, which makes many operations efficient. In most cases it distributes objects across bucket index shards evenly.
However this sharding scheme negatively impacts performance in two key areas – ordered bucket listing & resharding. Ordered listing requires a large number of read ops as it queries all bucket index shards to produce the listing. And the index resharding operation, used to right-size the number of shards as the bucket shrinks & grows, touches all shards.
Ordered bucket indices addresses these performance challenges. Keeping objects lexically ordered across shards makes ordered listing trivial. And resharding becomes a “local” operation, resharding only small sets of consecutive shards into more or fewer shards.
RGW users whose software stacks make extensive use of ordered bucket listing or ephemeral buckets (i.e., those that are created, receive a lot of objects in a short period of time, and then perhaps torn down), such as big data analytics, will learn about the sub-project & performance benchmarks.
Ordered Bucket Index Shards in RGW - Eric Ivancich, IBM
RGW on RADOS uses hash-based sharded bucket indices to store metadata associated w/ objects. The key advantage is that mapping an object name to a shard is constant-time, which makes many operations efficient. In most cases it distributes objects across bucket index shards evenly.
However this sharding scheme negatively impacts performance in two key areas – ordered bucket listing & resharding. Ordered listing requires a large number of read ops as it queries all bucket index shards to produce the listing. And the index resharding operation, used to right-size the number of shards as the bucket shrinks & grows, touches all shards.
Ordered bucket indices addresses these performance challenges. Keeping objects lexically ordered across shards makes ordered listing trivial. And resharding becomes a “local” operation, resharding only small sets of consecutive shards into more or fewer shards.
RGW users whose software stacks make extensive use of ordered bucket listing or ephemeral buckets (i.e., those that are created, receive a lot of objects in a short period of time, and then perhaps torn down), such as big data analytics, will learn about the sub-project & performance benchmarks.
