SNIA SDC 2025  - Model Training in Public Clouds: Case for IBM Storage Scale @SNIAVideo
SNIA SDC 2025  - Model Training in Public Clouds: Case for IBM Storage Scale  @SNIAVideo
Uploaded November 2025 | Updated November 2026, 11 hours ago
Over the last few years IBM has built several AI-HPC clusters to experiment with and train its Large Language Models (LLMs). One of the clusters – Cloud Vela – is especially notable because it explored less conventional approaches for building HPC clusters. Vela relies on public cloud infrastructure, runs in VMs, uses Ethernet, and relies on container orchestrator (Kubernetes) to manage workloads and resources. Training jobs submitted by data scientists produce two types of I/O traffic: 1) reading training data and 2) writing large periodic checkpoints. To satisfy the I/O requirements of Vela we architected a tiered storage system that runs a distributed file system (GPFS) on top of cloud object storage. In my presentation I will start with an overview of Vela’s design, how data scientists interact with it, and what workloads they run. I will then dive in the details of the storage system architecture and implementation for Vela. I will describe the performance and semantical challenges of the native cloud storage options and why we resorted to using a distributed file system over object storage. I will cover the design of our data mover and the solution integration w/ Kubernetes that makes it easy for the end user to provision file system volumes backed by object buckets without administrator intervention. I will also talk about our challenges related to observability and cache sizing.
Learn what I/O workloads distributed training jobs create and what storage requirements they have Learn a practical and cost-effective approach for architecting tiered storage architecture for AI training jobs : Learn an example of using public clouds and cloud-native technologies to run storage for training.
Presented by Vasily Tarasov, IBM
Learn More:
• SDC Website: snia.org/sniadeveloper
• SNIA Website: snia.org
• SNIA Educational Library: snia.org/library
• X: twitter.com/SNIA
• LinkedIn: linkedin.com/company/snia
SNIA SDC 2025  - Model Training in Public Clouds: Case for IBM Storage ScaleSNIA SDC 2025  - Panel Discussion: Do We Disrupt GPUs?SNIA SDC: StorageAI 2026 - Storage For AI 103, An Introduction to Storage for InferenceDNA data storage: from molecular memory to scalable realitySNIA SDC 2025  - Beyond Throughput: Benchmarking Storage for the Complex I/O Patterns of AIData Protection in the Ransomware EraEffect of different cations on controlling the orientation of chiral double-L DNA origami on solid.SNIA Partner Update - INCITS - June 2024DNA Data Storage on the goSDC: Updating Attendee Access Email and Sending LoginsHow The Storage Industry Can Create a Sustainability RoadmapSNIA SDC 2025  - Design Specification and AI-Driven Digital Twin Architecture for Storage Devices
SNIAVideo |

SNIA SDC 2025 - Model Training in Public Clouds: Case for IBM Storage Scale

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER