Uploaded November 2025 | Updated September 2026, 3 weeks ago
Scaling RGW D4N Caches With Experimental Offloading - Jesse Williamson & Matthew Benjamin, IBM
As workloads change and applications migrate to S3, RGW clusters find themselves taxed in new ways. RGW’s (experimental) D4N intermediate cache presents new opportunities to compensate for workload saturation in a range of scenarios.
In this presentation we use D4N caching as a setting for bucket-index acceleration with external indexes—-as a discrete caching accelerator, but also as a source of practical insight into use of external indexes in native, RADOS-based RGW clusters.
The talk will be approximately 25-30 minutes followed by a QA session. It will be presented in three sections: first, we’ll show curated examples of problematic situations such as scaling workloads with many record updates and deletions. Next, we'll consider what the characteristics of an improved solution would be, with emphasis placed on scaling and strong consistency. Finally, we'll show how our prototype work aims to achieve this by offloading D4N caching to a specialized system.
Audience: Ceph developers, users, and site administrators interested in Ceph object storage performance, esp. Metadata performance and performance improvement.
Scaling RGW D4N Caches With Experimental Offloading - Jesse Williamson & Matthew Benjamin, IBM
As workloads change and applications migrate to S3, RGW clusters find themselves taxed in new ways. RGW’s (experimental) D4N intermediate cache presents new opportunities to compensate for workload saturation in a range of scenarios.
In this presentation we use D4N caching as a setting for bucket-index acceleration with external indexes—-as a discrete caching accelerator, but also as a source of practical insight into use of external indexes in native, RADOS-based RGW clusters.
The talk will be approximately 25-30 minutes followed by a QA session. It will be presented in three sections: first, we’ll show curated examples of problematic situations such as scaling workloads with many record updates and deletions. Next, we'll consider what the characteristics of an improved solution would be, with emphasis placed on scaling and strong consistency. Finally, we'll show how our prototype work aims to achieve this by offloading D4N caching to a specialized system.
Audience: Ceph developers, users, and site administrators interested in Ceph object storage performance, esp. Metadata performance and performance improvement.
