Hybrid Storage is a term used to describe storage systems that are designed with a blend of flash-based solid state disk and hard disk drives to provide high performance at an affordable price. They try to address the dollar-per-IOPS vs dollar-per-GB decision that IT professionals have to grapple with. In addition they try to answer the question of how to best leverage flash by using flash as a large caching area, automatically moving the most active data into memory-based storage.
The Components of Hybrid Systems
The component of a hybrid system that gets most of the attention is the flash layer. This makes sense since the high performance part of the hybrid promise relies on being able to serve a large majority of the data from this memory-based storage area. Hybrid storage system vendors will try to differentiate based on their ability to provide cache accuracy. There are several ways that they can accomplish this.
First, they can provide excess flash capacity. The greater the percentage of flash in relation to overall capacity of the hard disk storage area the greater the likelihood of data being served from cache. This is an expensive and potentially wasteful method for achieving a high cache hit rate but is conceivably the easiest path to getting there.
Increasing flash capacity is not just limited to providing additional real capacity, some of this increase can be achieved by leveraging capacity optimization technologies like deduplication and compression of the flash area. As Storage Switzerland discussed in a recent report analyzing the return on investment of deduplication, applying these capacity efficiency technologies to flash related solutions can have a significant payoff.
While the effective cache capacity will vary between environments, 3X the actual physical size is a safe estimate and in many environments, a 5X to10X improvement is realistic. This means that a 1TB flash area could potentially act like 5TB, bringing potentially a 5X or greater chance of a cache hit. This caching would need to be done inline so that flash capacity could be always optimized.
Finally, the hybrid storage vendors can improve the technology surrounding the flash implementation to increase cache performance. This goes beyond improving cache algorithms themselves and looks at the way the cache is implemented.
For example most vendors only support cache reads, citing that if a cache layer fails their systems won’t risk data loss because all changed data is on the hard disk. While this is a fair concern it does create a problem that all write I/O must go to the hard disk which, of course, induces latency. More importantly, much of the reliability concern can be addressed through better storage system design.
Most of the concern behind this read-only precaution is probably “old world” thinking based on a time when volatile DRAM was the only option for the cache area. Since flash is nonvolatile there’s not the immediate risk of cache loss on power failure. Flash then merely needs to be protected from device failure just as the hard disk tier is protected. The flash devices themselves provide some protection thanks to their advanced controller technology. Finally, placing these in either a mirrored or RAIDed configuration completes the protection process.
Another concern about using flash for write traffic is that it could prematurely wear a flash device out. As Storage Switzerland discussed in the article “One Server SSD for the Enterprise” flash storage providers have gone to great lengths to make sure that flash devices can withstand almost any use case, including a high write environment like a write cache.
In addition, other technologies can help. For example, if inline deduplication is being performed more than likely the initial deduplication examination is happening while the data is still in DRAM. This process helps longevity in several ways. In many situations the write does not have to occur at all because the data already exists on the system. Also, no parity or second copy of that data needs to be created and written as part of the data protection scheme. Eliminating 5X the write traffic via deduplication can potentially make the flash cache area five times more durable. Not only does integrated deduplication help with cache longevity it also increases flash performance because all of these writes are eliminated.
With deduplication in place, data is first analyzed in DRAM for redundancy. If the data is ‘net new’ then it’s written to the flash storage area and the write acknowledgement is sent to the application. Flash is much faster than waiting for the hard disk system to record a write. Again the write is still secure. Flash memory is nonvolatile and hybrid systems that use this approach also have built in redundancy for the flash storage area.
Another performance boost from write caching comes into play when a hybrid storage system knows how to logically dump data from this cache. Some of these systems have the ability to logically order how data is written to the hard disk. This reduces the random nature of the I/O stream. Hard disks perform much better with sequential streams of data than random I/O. It also makes loading the read cache in the future more efficient since that data will be located contiguously when becomes hot and can be loaded into cache with fewer platter rotations and head movements.
The Hard Disk Layer
The other layer of the hybrid storage system is the hard disk drives themselves. The goal with this layer is to make them as attractive as possible from a cost per GB perspective. These can be done with two methods. First, large capacity hard drives can be used to minimize the cost per GB. Today 2TB drives are commonplace and 3TB drives are being introduced, and it won’t be long before 4TB drives are the standard. With each increase in capacity the price per GB comes down substantially.
The second method to drive down the cost of secondary storage is leveraging deduplication. The relatively low starting price per GB that these large capacity drives bring means that this data reduction efficiency won’t be as rewarding on an incremental basis as it has historically been. But because of the gross capacity savings it can still be significant.
Some hybrid systems avoid deduplication at the hard disk tier for fear of performance bottlenecks and a reduction in scalability. As Storage Switzerland discussed in a recent webinar “What is Breaking Deduplication” these challenges can be overcome by more efficient hash table management and reduced hash table size. With the correct implementation method deduplication on hard disk can be implemented in such a way that there is no noticeable performance impact.
Hybrid Storage System Futures
Faster Caching
Of course there is always a demand for greater performance and leveraging DRAM to handle more of the read and write cache responsibilities would be an ideal way to deliver it. While flash is significantly faster than hard disks, DRAM is significantly faster than flash, especially on writes. DRAM also doesn’t have the endurance or ‘wear out’ issues that flash does. But, the challenge is that unlike flash, DRAM is volatile and won’t survive a power failure, so using it to cache uncommitted writes is risky.
What’s needed is a nonvolatile alternative, one that’s now available in the form of NVDIMMs. These are DRAM modules that also have flash memory and a capacitor installed on them. When power is lost the capacitor provides enough charge for DRAM to dump its contents to flash. When power is restored the DRAM is quickly refreshed from flash. This makes NVDIMM an ideal write cache area and, as Storage Switzerland discussed in the article “How To Solve the SSD Endurance Problem”, can further prolong the life of the SSD tier.
Creating a faster HDD Tier
Even with a highly optimized flash area at some point data with have to be read from and written to HDD. When that occurs the latency difference between a DRAM enhanced flash storage area and a high capacity disk drive is significant. Something will need to be done to close this gap and increase the performance of the HDD tier.
The only truly effective method for improving hard drive performance is to have more of them; the more spindles there are the more I/O requests that can be serviced in parallel. The problem is that most hybrid systems have chosen an appliance approach, which can limit capacity scaling options. This choice makes sense as it keeps costs down and is simpler to implement but does limit the ability to add hard drives to address performance.
To keep the simplicity but overcome the limitations of low spindle counts hybrid storage system providers should consider using the smaller form factor 2.5″ disk drives. This would allow them to double or even triple the amount of hard drives installed in a system.
Seagate for example just started shipping its new Savvio 10k.6 2.5″ disk drive. These 10K RPM drives handily out perform the high capacity 7200RPM drives commonly used in hybrid systems and give the supplier the ability to provide two to three times as many spindles which also improves overall performance. The combination would narrow the performance gap between the flash tier and HDD tier allowing the storage system to provide excellent “cache-miss” performance.
Summary
Hybrid storage systems are likely to become the next wave of storage solutions that medium to large sized data centers use for most of their business critical workloads. Because these systems are built from the ground up to leverage flash they can strike the right balance of price and performance. Not all hybrid storage systems are created equal and this article should give some guidance as to what to look for in this important market. Most important is to understand the vendor’s cache optimization strategies and how severe of a impact will non-cache read or write be.
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 