Violin Memory Systems has started a marketing campaign pronouncing the death of hard disk drives. They are quick to qualify that disk is dead in terms of primary storage. Violin even had some cool tombstones made up. I want to agree with the claim, mostly because I want one of those tombstones. But I just can’t get there, based on their logic, and I don’t think that most of the data center managers that I talk to can either.
The Problem With the ‘Disk is Dead’ Logic
Here is the problem, primary storage has two distinct data types. The first type, transactional data, is absolutely ready for an all-flash world. And, I would agree whole-heartedly that this data is not only serviced better by flash, the management of those environments is also greatly simplified. If this were the only type of primary data, then I would agree with my friends at Violin, but it’s not. There is another type of primary data that most of the time has no business being on flash storage and can be stored far more cost effectively on a modern on-premises disk architecture or in the cloud.
The second primary storage data type is made up of user data, sensor data and machine generated data, also know as the Internet of Things (IoT). One could argue that this data is not primary but I think most users would agree that the files they are currently editing or modifying are indeed “primary”. The sensor and machine generated data is also typically stored on primary storage, at least initially. Sometimes, for processing, that data is later moved to flash storage, but that is typically driven by a specific user request more so than an ongoing operation. And yes, we know of situations where this data is stored on flash all the time, but that is the exception more than the rule. Most often, this data is better served by a non-flash tier.
Generally, this second type of primary data is sequentially stored and accessed, something that disk drives are actually quite effective at. Also, this data is often accessed by systems or devices that don’t have the sufficient processing or network connectivity to take advantage of flash. But this data does need some level of performance and reliability beyond what a disk archive, non-primary storage, would deliver.
The Disk is Dead Solution
In our recent webinar “Using the Cloud to Create a Truly All-Flash Data Center” Avere CEO Ron Bianchini and I discussed the challenge of creating an all-flash data center, which would be required to officially pronounce disk dead. We also covered the greater challenge of how to move data back and forth between these two types of primary storage. As we discussed in the webinar, that movement is really the big issue. If we could move data between tiers such that data being accessed from an application or by a user was always on flash, having two types of primary storage would be a non-issue. The reason for the all-flash data center mantra is that many data centers haven’t discovered how to effectively manage that data movement in an automated fashion. Going all-flash for all primary data is giving up.
If the data center adopts one of the options we present in the webinar, putting this second form of primary storage in the cloud, then it could pronounce disk dead for itself. But the cloud would still use disk for this data and the data center should still consider this data to be a part of primary storage. The only difference is that this second form of primary storage is in the cloud.
All-Flash data center proponents and those that are pronouncing the death of disk are basing these assumptions on how cost effective flash is for primary storage. Again, for the first form of primary storage, transactional data, I agree—put it on all-flash and be done. For the second form, all-flash vendors are counting on the price decreases in flash media, plus deduplication and compression to win the day. There are situations, especially when the second form of primary data is relatively small, that all-flash does win. But for most environments this data set is simply too large to put on flash, especially when there is limited benefit to doing so.
Given the advancement in drive capacities and data protection schemes, the cost gain that flash vendors claim to be making needs to be re-calculated. Once that is done, I think we will find that the practice of using hard-disk for primary storage will live on for a long, long time to come.