Expanding and pushing a 40GB/s capable cluster to the limit! @45Drives
Expanding and pushing a 40GB/s capable cluster to the limit!  @45Drives
Uploaded August 2025 | Updated November 2026, 1 day ago
🚀 Ceph Cluster Expansion & Benchmark – From 5 to 6 Nodes with Massive Performance Gains!

In Part-3 of our Build, Bench & Expand series, we’re taking our 5-node Ceph cluster to its absolute peak… then making it even bigger. You’ll see real-world multi-threaded, multi-client Ceph benchmark results, watch us push over 40GB/s throughput across the network, and witness just how easy it is to expand your cluster to 6 full F16 NVMe nodes.

Chapters:
00:00 – Welcome to Part 3: The Expand Phase
00:07 – Multi-Threaded, Multi-Client Benchmark Setup
01:12 – Recap: Build & Tune Phases (Parts 1 & 2)
02:12 – Why Single-Thread Tests Aren’t Enough
02:33 – Benchmark Environment & Hardware Specs
03:56 – VM Setup, RBD Mounting & ZFS Configuration
05:54 – Using Ansible to Run Benchmarks Simultaneously
07:19 – Benchmark Parameters & Direct I/O Importance
09:04 – Live Benchmark Run (40GB/s+ Over the Network!)
11:00 – Performance Breakdown: Throughput & IOPS
12:46 – Preparing to Expand the Ceph Cluster
13:43 – Production-Safe Benchmarking Practices
15:01 – Adding the 6th Node (F16 Server with NVMe)
17:21 – Automatic OSD Detection & Backfilling
18:29 – Data Rebalancing Across the Cluster
18:49 – Speeding Up Recovery with mClock Profiles
21:10 – Cluster Fully Rebalanced – Even Data Distribution
25:55 – Dashboard Overview After Expansion
26:58 – Recap & What’s Coming in Part 4

Whether you’re into homelabs, high-performance storage, or Ceph tuning, this is where theory meets jaw-dropping performance.

🔥 What You’ll See in This Video:

Maxing Out a 5-Node Ceph Cluster – 30+ clients hammering the system simultaneously.

Ceph Benchmark Numbers That Impress – ~40GB/s average reads, 11GB/s peak writes, and over 2 million IOPS.

Real-World Setup – 200Gb networking, AMD EPYC-powered Proxmox hosts, Ceph on RBD, and direct I/O tests.

The Expansion Phase – Adding a new node with just a few clicks in the Ceph dashboard.

Pro Tips for Ceph in Production – Safe tunings, avoiding risky shortcuts, and keeping performance AND stability.

Live Walkthrough – How Ceph automatically rebalances and distributes data after expansion.

💡 Why This Matters:
If you’ve ever wondered how Ceph scales when you throw more nodes, more NVMe drives, and more bandwidth at it—this is your answer. Ceph is designed for linear scalability, and we show you how that plays out in a real cluster under extreme load.

We also go behind the scenes on how to:

Use Ansible for simultaneous multi-client benchmarking.

Adjust mClock profiles for faster rebalancing.

Maintain production-ready best practices while chasing big performance gains.

🔥 Performance highlights before expansion:

Sequential Writes: Peak 11GB/s, Avg 8GB/s.

Sequential Reads: Peak ~50GB/s, Avg 40GB/s.

Random Writes: 408K IOPS peak.

Random Reads: 2.07M IOPS peak, 1.5M avg.

🔗 Enterprise Data Storage & Virtual Compute Built on Open Platform& Open Source: 45drives.com/products/hardware-storage-and-virtualization-servers

At 45Drives, we understand that enterprise customers value solutions that prioritize risk mitigation while being reliable, secure, performant, scalable, and high-capacity.

📲 Follow Us for More:
LinkedIn: linkedin.com/company/45drives
Instagram: instagram.com/45drives
Facebook: facebook.com/45drives
X: x.com/45drives

We post tips, server builds, and behind-the-scenes peeks you won’t see on YouTube.

✅ Subscribe & Watch the Series:
This is Part 3 of our Build, Bench & Expand journey:
PART-1: youtu.be/9tqCJPnecHw?si=Hd3X-w0F_OjdECSm
PART-2: youtu.be/2PQUYdxUwn8?si=fly-bxe38SfgYUOU
SUMMARY OF PART-2: youtu.be/MfsKn00OzDY?si=TE0w5Okf8yJK-6QB


Hit Subscribe so you don’t miss the final part – and more deep dives into Ceph, ZFS, Proxmox, and Enterprise infrastructure.

#ceph #enterprisestorage #storageserver #nvme #100GbNetworking #proxmox #zfs #IOPS #ClusterExpansion #45drives #storagesolutions #techtips #networkspeed
Expanding and pushing a 40GB/s capable cluster to the limit!Ransomware Protection with SnapShield: The Last Line of Defense featuring @LAWRENCESYSTEMSUnderstanding Software Defined Networking with @ProxmoxVE - Combining Virtualization and StorageIntroducing the Stornado E16: 45Drives Fastest Gen5 All-Flash Enterprise Storage Server@Cephstorage Snapshot Strategies to Protect Your Data #CephFS45Drives - Our Analogy for ZFS WritesProtect Your Storage from Fire, Floods & Meteors!45Drives Creators Summit 2025 - Official Livestream from WILMINGTON, NC100% Uptime for Less: Building a 2-Node #DRBD Highly Available Cluster with Pacemaker/CorosyncWhats Spinnin with Dave Dickerson from CIQ/Rocky Linux? Season 2 Episode 5 #livepodcastThe Ultimate, Compact Hyperconverged Virtual Compute & Storage Solution? The Proxinator VM2!45Drives - Introducing the Stornado F16: 16-Bay All-Flash Server for NVMe (or SATA & SAS)
45Drives |

Expanding and pushing a 40GB/s capable cluster to the limit!

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER