Flash storage is making an enormous impact on the data center. Armed with a flash storage system, IT professionals are creating virtual environments with much higher VM to host ratios and are designing databases to support many more users and transactions. There is little debate that the future of flash is bright. But we still have to see which flash storage solutions will be the ultimate winners. In the first entry of this series we will look at the Future of Hybrid Storage.
A little History
Performance storage changed dramatically in the last decade. Ten years ago, most high-performance storage systems were built from hard disk drives. To meet performance demands data was wide-striped across dozens if not hundreds of hard drives. In many cases, those drives were formatted so that only the outer edge and higher performing sections of the platter stored data. In extreme cases organizations used DRAM, not flash, storage solutions from companies like Texas Memory Systems (now owned by IBM), Kaminario and Violin to solve, performance problems, but those solutions were extremely expensive. Flash brought down the price of performance, so much so that within a few years of introduction to the enterprise it was the most cost-effective way to meet the ever increasing demand for storage performance.
The increasing Use Case for Flash
There are three primary use cases that cause a data center to invest in a flash storage system; virtual desktops, virtual servers and databases. These environments have I/O profiles that justify the investment in flash storage. But with the introduction of Multi-Level Cell (MLC) and now 3D NAND Triple Level Cell (TLC) the use case for flash is expanding. The debate over what data should be on flash is essentially over; all active data should be on flash storage.
The Future of Hybrid Storage
The recommendation to store all dynamic data on flash, however, is not necessarily a requirement that every data center has to purchase an all-flash array. Hybrid storage systems have a future.
First, while 3D NAND TLC is less expensive it is not as reliable, using MLC to act as a shock absorber to TLC seems like a perfect match. Hybrid storage systems are in an ideal position to take advantage of an MLC/TLC combination.
Second, hard drive storage is no longer stuck at 4TBs. With the introduction of helium and SMR drives, hard disk storage manufacturers are currently delivering 8TB and 10TB drives with 20TB drives possible soon. Hard drive technology will still continue to be less expensive per GB than flash storage for the foreseeable future. Hybrid systems that provide a large flash tier with a larger hard disk tier should be able to ensure the predictable performance that all-flash vendors claim to be their advantage.
The big challenge for hybrid vendors is that they typically design systems as a two-tier architecture. The data center will for the foreseeable future need three tiers; a small but reliable MLC flash tier, a large TLC tier and a vast HDD tier leveraging 8TB or 10TB hard disk drives. Not only will they need to improve their software to support these multiple layers, they also need to fine tune their software for the unique attributes of TLC flash and SMR drives.
Most IT professionals understand that most data is inactive. A hybrid system seems more logical to them, but they are selecting all-flash arrays because they are concerned about the potential performance drop during a cache or tier miss. Hybrid flash systems benefit from the same price decreases in flash that all-flash systems do, so they should be able to extend the size of their flash tiers to the point that the chances of a cache miss are so rare the temporary performance impact of fetching data from a hard disk drive is negligible.
All-Flash array vendors continue to drive down the cost of their systems and several have the ability to backup or archive to an object storage-based secondary storage system. We’ll explore the future of all-flash arrays in our next installment of the ‘The Future of Flash Storage’.