Innovative Storage Solutions Showcased to Match Accelerated ML Workloads
In an era marked by rapid technological strides in machine learning (ML) systems, storage solutions are evolving to meet the burgeoning demands of faster accelerators and larger datasets. MLCommons®, an organisation dedicated to developing benchmarks for ML performance, has recently unveiled the results of its MLPerf® Storage v1.0 benchmark suite. This industry-standard suite is designed to evaluate the performance of storage systems specifically for ML workloads, ensuring that storage keeps pace with the rapid advances in compute technologies.
The Benchmark's Unique Contributions
The MLPerf Storage benchmark distinguishes itself as the first open and transparent benchmark designed to measure storage performance across a variety of ML training scenarios. The suite simulates the storage demands encountered in several ML training scenarios, employing a diverse set of models, accelerators, and workloads to create accurate storage usage patterns. This allows it to precisely measure a storage system's ability to keep up with compute needs without necessitating the actual execution of training sessions, thereby making it more accessible.
The benchmark includes three models to represent different AI training patterns: 3D-UNet, Resnet50, and CosmoFlow. These models vary in sample sizes, from hundreds of megabytes down to hundreds of kilobytes, and feature different simulated 'think times' ranging from milliseconds to hundreds of milliseconds, to replicate real-world usage.
Moreover, the benchmark emulates NVIDIA A100 and H100 models, reflecting current accelerator technologies. Notably, the H100 is able to reduce the per-batch computation time for the 3D-UNet model by 76% compared to the older V100 accelerator, altering the workload's requirements from bandwidth-sensitive to latency-sensitive.
Impact of Distributed Training and System Architecture
Support for distributed training is a significant aspect of the MLPerf Storage v1.0. Distributed training, a common practice for accelerating ML model training across large datasets, poses specific challenges in delivering high throughput and serving multiple nodes simultaneously. This is a crucial scenario for storage systems to handle, reflecting the complexities of real-world large-scale ML training environments.
“The submissions we received for the v1.0 benchmark revealed a broad array of innovative approaches to providing high-speed, high-scale storage,” commented Oana Balmau, co-chair of the MLPerf Storage working group. She noted that no single 'best of breed' architecture emerged, illustrating the diverse methods employed to meet the high performance demands of ML workloads.
Performance Gains and Industry Participation
The MLPerf Storage v1.0 benchmark results underscore the advancements in storage technology. The diversity of workloads submitted highlights the industry's recognition of ML workloads' importance across different storage solutions.
Curtis Anderson, co-chair of the MLPerf Storage working group, pointed out that the new, faster accelerator hardware has significantly raised the performance standards for storage solutions. "System architects must procure the fastest storage available to prevent expensive accelerators from sitting idle," he said, adding that storage providers are rising to this challenge with innovative designs.
The v1.0 benchmark includes over 100 performance results from 13 organizations, showcasing contributions from both large and small players in the sector. Participants include DDN, Hammerspace, Hewlett Packard Enterprise, Huawei, IEIT SYSTEMS, Juicedata, Lightbits Labs, MangoBoost, Nutanix, Simplyblock, Volumez, WEKA, and YanRong Tech.
David Kanter, Head of MLPerf at MLCommons, expressed his enthusiasm over the wide participation in the groundbreaking v1.0 Storage benchmark. He emphasized that the industry is actively working to ensure that storage technologies evolve in tandem with advancements in AI. The benchmark results provide a valuable reference for both technology providers to meet performance targets and consumers to optimise their ML training systems.
Conclusion
The introduction of the MLPerf Storage v1.0 benchmark represents a significant step forward in evaluating and improving storage systems for ML workloads. By providing a transparent, open, and reproducible method to measure storage performance, MLCommons is driving innovation and ensuring that storage systems can keep up with the accelerating pace of AI advancements.
Source: Noah Wire Services