The raw performance of a single enterprise flash drive is often higher than 70,000 IOPS. Most all-flash arrays, even though vendors fill them with 12 to 24 flash drives, deliver only a few hundred thousand IOPS. The cause of the sudden drop in performance is in most cases the storage software. The overhead introduced by the storage software leads many organizations to only use flash when it is directly attached to the server, which saves them both network and storage software overhead. The storage software overhead problem promises to get worse because media continues to get faster and densities continue to increase.
What to Do About Storage Software?
Eliminating storage system software by only using direct attached storage doesn’t increase software efficiency. The operating system software adds plenty of overhead on its own. Also, the advent of NVMe over Fabrics promises to remove most of the latency introduced by networking. Another challenge with the elimination of storage software is the features it delivers are considered required capabilities.
One approach is to move the storage software into the hardware. Several vendors on the market have moved portions of the storage operating software onto field programmable gate arrays (FPGA), and a few have placed it on ASICS. These approaches should significantly improve storage operations while maintaining most enterprise features.
The other option is to re-write the storage software from the ground up. Most storage system software is over 10 years old, and much of it is over 20 years old. In that time new methods to develop software are available, and of course, flash is an expected storage device. The processors that drive software are also different. Intel processors show the majority of their performance gain because they are multi-core not because the raw speed of the processor has increased.
Starting from scratch instead of trying to retrofit code that is a decade or two old should by itself improve performance. Also, it gives developers the opportunity to make their code genuinely multi-threaded, equally taking advantage of all available cores.
Starting over also gives developers an opportunity to rethink the various algorithms that calculate capabilities like volume expansion, protection from media failure (RAID) and snapshots. It also enables them to re-think how to implement protocols like block, file, and object. Developers have the opportunity to integrate these protocols instead of bolting them on.
Conclusion
Storage Software is the next great battleground for vendor differentiation. The need to deploy dozens of flash drives to derive performance is both expensive and inefficient. Storage software can no longer hide behind highly latent storage media like hard disk drives or inefficient protocols like SCSI. Flash and NVMe remove the latency and inefficiency of the storage architecture, now all fingers point to the storage software as the reason for lack of performance.
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 