The storage architecture requirements for Commercial HPC are different from those for traditional enterprise storage architecture. They are also different from the traditional HPC requirements. Most enterprise IT professionals try to either extend their current storage architectures, which are supporting databases and virtual machines, to meet the IO demands of Commercial HPC. But they find that these systems lack the necessary scalability and the ability to deliver cost-effective performance to the mixed HPC workload. Sometimes enterprise IT professionals attempt to leverage traditional HPC storage architectures, but the do-it-yourself nature of these systems are too time-consuming to implement and difficult to support.
The first step is to understand the priorities of the enterprise. Academia often overlooks the DIY nature of traditional HPC architectures because one of their primary priorities is to cut costs. Academia also tends to have more time to work through implementation and support challenges. While enterprises share cost concerns, there is greater emphasis on time to value in Commercial HPC. Most of these organizations already have massive amounts of data to be analyzed and have more being added to it on a daily basis. They need systems that can plug in and start solving problems from day one.
The second step is to differentiate between a bundled system and an integrated system. There are HPC vendors that have bundled together open source HPC file system software and commodity hardware to create a system that appears to be turnkey. While these systems may help improve the implementation experience, they suffer from the same complexities as other HPC storage systems. Day to day management and cluster expansion are still a do-it-yourself experience. An integrated Commercial HPC system stays consistent, providing a turnkey implementation, operation, and expansion experience.
The final step is migrating data to the new Commercial HPC storage system. It is crucial that the system selected support legacy protocols like NFS and SMB, as well as performance oriented parallel access protocols. Legacy protocols allow the more traditional enterprise storage systems already in place to gradually migrate data and remain online until the operations on the new system can be confirmed.
StorageSwiss Take
The demands of Commercial HPC storage are different from both traditional enterprise storage and traditional HPC storage. The priorities of the organization are also different, time to value vs. cost savings, which leads many organizations to a turnkey HPC system. The type of ideal turnkey system is different, however, and many organizations need an integrated system. Finally, the migration strategy is different because so much of the data to examine already exists.
To learn more about Designing Storage Architectures for Commercial HPC register for our on demand webinar “Is Your Storage Ready for Commercial HPC? – Three Steps to Take“. In it we discuss what Commercial HPC is, the challenges enterprises face as they create a Commercial HPC Architecture and we’ll provide more detail the three steps organizations should take in developing their HPC storage strategy.
Watch On Demand
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 