Originally, hybrid flash arrays solved a price problem. Hybrid Flash 2.0 solves a performance problem. In their first iteration, these systems were designed to bridge the gap between expensive flash storage and more affordable hard disk drive storage. But as all-flash arrays came to market and the price came down, the need for hybrid systems became less obvious. Is there still a need for a hybrid flash storage system?
All the development time hybrid vendors invested in deciding when and how data should move from flash to hard disk won’t go to waste. Technologies like NVMe and high density flash are changing the way storage systems need to interact with flash media.
Hybrid for NVMe
NVMe is a high performance storage protocol designed specifically for flash media. It stacks up more commands and creates a higher queue depth than traditional SCSI protocols can. Because memory-based media does not have rotational latency like a hard disk does it can process the increase in commands and deeper queues, as a result delivering significantly better performance than the Serial Attached SCSI (SAS) alternative.
The problem is building an entire all-flash array with nothing but NVMe drives is expensive. Both the NVMe interconnects and the drives themselves come at a price premium, at least today, compared to their SAS attached brethren. And for most data centers the technology is also overkill for the large majority of their workloads.
A Hybrid Flash 2.0 system should offer a split bus where the most active IO is sent down an NVMe path, but as data ages it is migrated to SAS attached drives and leverage SCSI protocol. The split bus design enables storage system vendors to keep costs down and offer more realistic performance to their customers.
Ultra-High Density Flash
Beyond performance one of the more compelling aspects of flash technology is how much capacity can be squeezed into a storage system without concern for heat or vibration issues. Ultra-high density (UHD) flash modules today are over 16TBs in capacity and at least two vendors have announced that they will deliver 50TB+ of flash capacity per drive or module before the end of the year.
The problem with all this density is deciding how to use these modules. First, they don’t typically have the same endurance as high performance NVMe flash, often only allowing one full drive write per day. Second, even if endurance was not a concern putting all the organization’s data on a flash system with just high capacity flash would require a massive CPU and networking infrastructure in the array in order to keep up with them. Finally, to date high capacity flash modules are SAS attached, not NVMe attached.
Factoring in acceptable endurance, the difficulty of a single storage system driving PBs of flash storage and the limitations of SAS, these drives are best used for read heavy workloads. They make an ideal secondary tier for a set of NVMe drives front ending the storage system. A hybrid flash vendor, with some work, can create a storage system that leverages a split bus as well as a combination of NVMe and high capacity flash drives to provide data centers with a single system to meet all of their processing needs.
One More Thing
Hybrid Flash 2.0 also solves one other big challenge. The complaint with Hybrid Flash 1.0 systems was the potential for a performance drop if there was a “tier miss” and data had to be read from the hard disk tier. It was a legitimate concern because of the massive read performance differentiation between flash and hard disk drives. Hybrid Flash 2.0 does not have this concern, while read performance is lower, it is not as dramatically low as a hard disk drive. A primary tier flash miss is almost a non-event with Hybrid Flash 2.0
To learn more about Hybrid Flash 2.0, register to watch our on demand webinar, “Where is the All-Flash Data Center?” In this webinar you’ll learn why the all-flash data center has not materialized and how Hybrid Flash 2.0 may be the solution.