As Artificial Intelligence, Machine Learning and Deep Learning workloads go mainstream, organizations are struggling with how to develop a storage infrastructure to best meet the unique challenges of these workloads. AI/ML workloads typically include hundreds, if not thousands of servers accessing hundreds of terabytes of often represented by billions of small files. Since these workloads are trying to simulate or exceed human thinking, they need to traverse the data set very quickly.
The Storage Challenges AI and High-Velocity Analytics Present
The typical AI and high-velocity analytics workload require shared access to a storage system that provides high IO bandwidth, highly transactional IO and massive storage capacity. These environments tend to grow rapidly so scale-out architectures which add capacity and performance by adding additional storage servers (nodes) are preferred. Access to the data, however, can’t be bottlenecked by a single control node. Instead, the AI/ML workload needs parallel access to data so that no single node becomes the bottleneck.
In our eBook Storage Switzerland discusses the challenges of using legacy file systems with AI/ML workloads and what a modern file system built from the ground up for AI/ML should look like. We also look at the benchmarks that organizations use to evaluate these file systems. The eBook helps IT professionals decipher how vendors are achieving the various benchmark results and how those results relate to the real world.
To receive your eBook register for our fast paced 15 minute on demand webinar “Finding the Right File System for AI and ML Workloads.” In this webinar Storage Switzerland and WekaIO will review the various file-system options available to organizations looking to create storage architectures for AI, ML and analytics. The eBook is available as an attachment to the webinar.