While specialized all-flash storage systems grab the headlines, hybrid storage systems are increasingly becoming the workhorses of the data center, hosting the majority of applications, virtual machines and file shares. To expand the number of workloads that they can support many hybrid storage system manufacturers have turned to solid state disks (SSDs) to complement the hard disk drives in the system and boost performance. By using the SSD tier as a cache to the HDD tier, they’ve attempted to not only unify SAN and NAS storage protocols but also integrate the different storage media tiers.
The problem is that most of these hybrid storage systems treat all SSDs and the workloads that use them the same, offering limited choice and customization of how to take advantage of the high performance tier. In this article we will explain the different types of cache and how, in order, to best expand the number of workloads that a hybrid storage system can support, the cache tier should be flexible and tunable to the needs of the application.
As discussed above, hybrid storage systems typically use the SSD tier as a caching area to accelerate the most active data sets. This is an excellent way to automate the use and limit the capacity requirement of SSD. Because of its premium price optimal use of SSD is critical. But in spite of that requirement, most of these systems take a one-size-fits-all approach to leveraging the SSD. The reality is that there are two sides to I/O (reads and writes) plus each application has different demands of high performance storage.
Each workload, and even each side of the I/O equation, needs to be managed differently. Using the exact same SSD for both or ignoring one type of I/O may not be the best use of the technology. It is this lack of flexibility that leads organizations to consider point solutions for specific tasks. But if the cache can be properly designed with increased flexibility, the number of workloads that the hybrid storage system can support is expanded and the need to buy a point performance solution is minimized or eliminated.
Understanding Cache Types
The first step in developing a hybrid storage system that can support a wider range of use cases is to make sure it can sufficiently handle inbound writes. Providing high performance for write data is critical for workloads like virtual desktop infrastructures and databases, both of which tend to be write-heavy. To understand how writes can be better handled by the hybrid storage system requires a knowledge of the way data can be cached. The problem is that many systems either ignore writes or don’t cache them in an optimal fashion. Most use the SSD to only cache data that has been already written to hard disk and is being read again. In other words they only perform read caching.
Read-Only Caches
Since all flash memory has a finite life expectancy based on the number of write operations it sustains, the justification for a read-only cache is to avoid early cache burnout by making sure that only truly active data is being cached to the solid state area. Also, these caches are considered the safest since data is always written to hard disk first; failure at the flash layer would not result in data loss.
This would be a fair design consideration if these caches were all actually read-only, meaning that new data is never stored in flash until it has been qualified. But most hybrid storage systems have abandoned read-only caching in favor of write-through caching (described below). The reason is that it can take a considerable period of time and repeated accesses for a data set to ‘qualify’ for promotion to the cache tier. This process is called “cache warming” and until that qualification happens the overall performance of the system is dragged down by the speed of its hard disk storage.
Write-Through Caches
To get around the ‘performance penalty’ of a read-only cache many hybrid storage systems use a write-through caching technique. In this process data is written into cache memory AND to the hard disk layer simultaneously, but acknowledgement of that write doesn’t occur until the disk has completed its write operation. The application has to wait on the slower disk drives similar to the process involving a read-only cache.
In other words there is no write performance gain, but also no risk to data, as there could be if writes were held in cache before being written to disk. There is, however, a read performance gain. The chances of active data being in the cache area are greatly increased since all new data is captured there for a period of time. In other words much of the cache warm-up cycle is avoided.
The problem with a write-through cache, other than not delivering any performance gain on writes, is that it will wear out a flash-based SSD much faster than in a read-only cache. This is because all writes in a write-though cache go to the flash area at the same time that they go to the hard disk.
Write-Back Caching
The final type of caching technique is write-back caching, an asynchronous process in which the writes are acknowledged to the application as soon as they land in the cache area, prior to them be committed to the hard disk tier. This technique delivers the highest write performance possible but does so with some risks.
First, data that’s resident only in the cache is in a transient state. If there’s a power loss or a failure at the cache tier, data may never be committed to the hard disk even though the application assumes that it was. At a minimum this will lead to data loss but more likely it will lead to data corruption. Second, like the write-through scenario described above, this type of caching can wear through a flash based SSD faster than a read-only implementation. Finally, while faster than the HDD equivalent, flash based storage is dramatically slower at processing writes than reads. In other words, there is room for improvement.
Designing A Flexible Cache Technology
To make a win out of this “no-win situation” companies like Nexsan are deploying a flexible cache architecture that complements the hard disk tier and eliminates many of the problems described above. In doing so these systems can provide responsive performance to a wide variety of workloads from the same storage system.
For performance, write-back caching has a clear advantage, but storage system vendors need to design around the endurance limitations of flash memory and the potential risks of data loss. An ideal solution is to use a small DRAM-based SSD for inbound writes instead of a flash-based SSD. DRAM does not suffer the wear issues that flash does nor does it have the write penalty that flash does.
Since DRAM is more expensive than flash using it sparingly makes sense. DRAM is ideal for write caches because the capacity of inbound writes are relatively small in comparison to the potential data to be read.
DRAM does of course have one key challenge; it is volatile, meaning that if power is lost, so is all the data. Fortunately, this weakness can be overcome. For example, Nexsan has selected a SSD augmented with DRAM that has a capacitor to keep the DRAM area charged long enough for the data to be copied to a flash backup area on the SSD. And when power is returned the flash quickly restores its data to that DRAM area.
Using DRAM Cache To Prolong Flash Life
Another advantage of using DRAM as a write cache is that it can prolong the life of the flash-based read cache. Changes or additions of data often occur in clusters, with the same data segment being updated repeatedly within a few seconds or minutes. The DRAM can act as a buffer and coalesce these changes so that fewer updates are made to the flash storage area. This leads to fewer, larger writes to flash, delivering better life expectancy and better flash performance. If a hybrid storage system integrates these two components it could use less expensive forms of flash storage, like MLC, and still get better life expectancy than the premium grade flash storage.
Customizable Read Caches
Leveraging a combination of DRAM and flash to cache the most active data should lead to both improved performance and longer flash life. The next step is to provide fine tuning so that only certain data ends up occupying the cache. Some systems will allow the “pining” of data into cache but this requires file system awareness, which isn’t typically available for block storage protocols like iSCSI.
To get around this vendors like Nexsan are providing the ability to dedicate a Flash SSD for specific volumes or even excluding other volumes from the cache altogether. This allows storage managers to dictate how the premium memory based storage tiers are utilized. For example, a volume that serves up office productivity data to users on a WiFi network will not see any advantage to an SSD cache tier. But a volume that is dedicated to a virtual server or desktop infrastructure would have a lot to gain.
The system is flexible enough to even allow for 100% cache by configuring an SSD to be the same size as the HDD volume, something that could be ideal for database environments. The net effect would be to provide the performance of premium priced all-flash arrays with the protection of a hard disk back end.
Conclusion
The challenge facing hybrid storage systems is making sure that they can handle as wide a variety of workloads as possible. While most of these systems have leveraged some form of SSD to achieve that goal, most have not provided enough flexibility in the cache tier to fully exploit memory based storage. Companies like Nexsan, with their mixed DRAM/Flash tier plus the ability to customize caching at a volume level, allow storage planners to greatly extend the use of these workhorse storage systems, even to niche applications in the data center.
Nexsan is a client of Storage Switzerland
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 