At the FujiFilm Global IT Executive Summit there was a lot of discussion about data management and making sure the right data is on the right storage tier at the right time. Data management has been a core IT function for years. But a lot of vendors are suggesting you no longer need to tier data. The new message is you should simply put all of your data on a flash array and, voila, all your problems will go away. These vendors claim flash is not inexpensive enough to justify support for all types of data but the gains in operational simplicity will outweigh any additional expenditures on storage. If something sounds too good to be true, it usually is, especially in IT.
Fuzzy All-Flash Math
When an all-flash vendor claims price parity with a hard disk based array, you have to examine closely what they are comparing themselves to. The comparison is typically to a production class, high-performance hard disk system. This means 15K RPM drives and plenty of them. In that circumstance the all-flash vendors are absolutely correct. Flash is not only less expensive, it more than likely takes up less rack space. But simply installing flash may not solve all your performance problems, but that’s a blog for another day.
The “flash has reached price parity with disk” claim starts to fall apart when you compare flash storage to capacity hard disk and especially, dare I say it, tape. Now the flash vendors may say these comparisons are not fair, yet they are the ones claiming to can flatten your storage tiers down to a single tier.
Flat Storage?
There are two justifications to the flat storage tier strategy. The first is that flash is cheaper, which when compared to capacity devices simple math will show that it is not. The second is managing data is too hard and complex which means the additional cost of flash storage is justified by the simplicity it brings. Flash, under this justification, does not have to be cheaper than capacity storage it just has to be cheap enough that you can cost justify the simplicity.
The problem is that capacity storage, disk and tape, is significantly less expensive than flash. The other problem is that data management is not as hard as it used to be. There are turnkey solutions that can present object storage and tape storage as mountable network volumes and simply copy data to it. There are also plenty of solutions that will automatically move data from primary storage to secondary hard disk storage to tape storage. And LTFS, as we have discussed many times, provides an open tape format that can be read by virtually any operating system.
Tiers Work
Applying data management and leveraging the various storage tiers available to you leads to a more cost effective storage infrastructure. It also improves data protection, data retention, data security and data reliability. The key is to understand data management, something that Storage Switzerland covers in our on demand webinar “The Art of Data Management“.
Watch On Demand
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 
Well, Mr. Crump is right…storage tiers work. No one has an unlimited budget for stuffing all their seldom used and cold unstructured data in a flash array. And since old and cold unstructured data may account for 80 percent or more of your data, you really need to use management to tier all that data out to an object store.
The one product/service that I like in this space is Komprise. Komprise is easy to use and easy to setup. The Komprise “observer” analyzes all your SMB and NFS shares and helps you create plans for managing data that is old and cold so it can be moved from primary storage to object storage. You save money and keep the lid on adding/replacing your primary storage systems. The best thing is your users and their applications will still be able to easily locate their data because it will look like it is still where they expected to find it.
Call it what you want, but Komprise combined with Cloudian HyperStore is the smart way to manage your unstructured data.
Storage tiers work, if you have enough storage to justify the management cost. On one hand, tiering has been getting easier, as George said. But on the other hand, flash has been dropping in price/capacity faster than disk. The end result is that each year you need a larger amount of storage to justify doing tiering, and if your storage needs are below that growing threshold, it’s probably cheaper for you to just go all-flash rather than tier your data. (This doesn’t count the backup and archive storage, which should nearly always be on a cheaper tier made up of hard drives, tape, and/or a cloud service.)
In the beginning nobody could afford to go all-flash. Then some individual projects did, then department, then small businesses. Soon some medium-sized enterprises will go all-flash rather than tier their storage. Not sure if the large enterprises can afford to go all-flash anytime soon–they can still save a lot of money with tiering. And the big cloud and hyperscaler vendors are pioneering new ways to do tiering to disk, tape, or even optical.