Unintended Consequences – All-Flash Arrays are Shrinking the Storage Market

The consensus outcome of reports on the ‘state of the storage market’ shows a market that is in decline by at least 10 percent. Those same research firms indicate that overall data capacity is growing at an alarming rate. So why is the market shrinking? The answer is often overlooked: storage systems are significantly more efficient, in terms of both performance and capacity, than they ever were. All-Flash Arrays (AFA) are the primary catalyst for the gain in efficiency. A data center armed with an AFA needs fewer storage systems, can use capacity more efficiently and can expect at least twice the usable life out of their investment.

Performance Efficiency

Ten years ago a hard disk-based storage system capable of generating a few hundred thousand IOPS occupied multiple data center racks. The way to increase the performance of hard drive-based systems was to use hundreds of hard disks. To further enhance performance, those hard drives were formatted so that only a quarter of the capacity was available to the application. This “short-stroking” made sure that data landed on the fastest portion of the hard disk platter. The combination of using hundreds of hard disk drives and short-stroking them inflated the amount of capacity sold. Most hard disk-based arrays never reach 20 percent of their potential capacity.

Configured like this, the hard disk array could only support a few workloads. The rotational latency of hard drive based systems led to the purchase of multiple units, one for each environment in the data center. Hard disk array systems, because they were always under pressure to deliver more performance, were also quickly replaced when the next generation storage system came out. Vendors built their business models counting on replacing a customer’s system every three years.

Today, thanks to AFAs, a 2U system can generate over a million IOPS, even mid-range AFAs can generate over 100K IOPS. For some data centers, a single AFA can provide all the performance their entire data center needs as well as meet the performance demand for the next five years or more. Even if the system does not meet their long-term performance needs, most AFAs are more upgradable than their HDD brethren. While the concept of an upgradeable storage controller or a scale-out storage system is not new, the advantage that AFAs enjoy is that the media attached to that controller –flash – does not need to change to improve performance. With an AFA the flash is not the bottleneck, the controller CPU and the software are the bottleneck. An upgrade to either of these components improves performance without moving data.

Capacity Efficiency

AFAs make primary storage, deduplication and compression practical from a performance standpoint. Server or desktop virtualization makes it feasible from an efficiency gains perspective. Most data centers that we talk to that are using deduplication and compression are seeing a 50 percent or greater efficiency rate and the more virtualized they are the higher that percentage is.

Deduplication and compression are not the only reasons that flash arrays are more efficient. The performance of an AFA also leads to capacity efficiencies. There is no need to use hundreds or even dozens of solid state drives (SSD) to meet the performance requirement. There is also no section of the SSD faster than another section. An AFA allows the IT professional to buy the exact amount of flash capacity they need; it can deliver peak performance at almost 100 percent capacity. Most AFAs should be able to reach 70 percent or more of their potential capacity.

Mixed Workloads

While some AFAs are better at mixed workloads than others are, most of them, thanks to their excess performance, can support more than one workload type. The ability to better support mixed workloads should lead to less total storage systems in the data center. As we discuss in our article “Flash + Object – The Emergence of a Two-Tier Enterprise“, many data centers should be able to consolidate down to two systems; an AFA and a high capacity storage system for unstructured data.

StorageSwiss Take

The storage market is going through a temporary adjustment. Organizations are moving to AFAs or flash heavy hybrid systems, but those systems will be serviceable for a much longer period. Vendors and investors need to adjust their thinking and adapt to the new reality of storage.

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , , , , ,
Posted in Blog
One comment on “Unintended Consequences – All-Flash Arrays are Shrinking the Storage Market
  1. Keith Carpenter says:

    George, we hasn’t seen this level of disruptions in over 15 years. Not only will storage systems need less HDD spindles to meet the applications’ needs, but IT will need to buy significantly less storage controllers if Object performance isn’t the answer. This creates huge costs savings. Less heads means less Software Licensing and less Support Contracts. It means less footprint needing less power and cooling. When the storage software can effectively keep the active data set in DRAM and Flash, IT shops that have 25, 50 or 100+ storage controllers will be able consolidate as much as 10:1….similar to when they virtualized their server infrastructure a decade ago. And most importantly, as with many “disrupters”, this new cost structure will create new business use cases thought impossible before. We are just at the beginning….

Comments are closed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,220 other followers

Blog Stats
%d bloggers like this: