The consensus outcome of reports on the ‘state of the storage market’ shows a market that is in decline by at least 10 percent. Those same research firms indicate that overall data capacity is growing at an alarming rate. So why is the market shrinking? The answer is often overlooked: storage systems are significantly more efficient, in terms of both performance and capacity, than they ever were. All-Flash Arrays (AFA) are the primary catalyst for the gain in efficiency. A data center armed with an AFA needs fewer storage systems, can use capacity more efficiently and can expect at least twice the usable life out of their investment.
Ten years ago a hard disk-based storage system capable of generating a few hundred thousand IOPS occupied multiple data center racks. The way to increase the performance of hard drive-based systems was to use hundreds of hard disks. To further enhance performance, those hard drives were formatted so that only a quarter of the capacity was available to the application. This “short-stroking” made sure that data landed on the fastest portion of the hard disk platter. The combination of using hundreds of hard disk drives and short-stroking them inflated the amount of capacity sold. Most hard disk-based arrays never reach 20 percent of their potential capacity.
Configured like this, the hard disk array could only support a few workloads. The rotational latency of hard drive based systems led to the purchase of multiple units, one for each environment in the data center. Hard disk array systems, because they were always under pressure to deliver more performance, were also quickly replaced when the next generation storage system came out. Vendors built their business models counting on replacing a customer’s system every three years.
Today, thanks to AFAs, a 2U system can generate over a million IOPS, even mid-range AFAs can generate over 100K IOPS. For some data centers, a single AFA can provide all the performance their entire data center needs as well as meet the performance demand for the next five years or more. Even if the system does not meet their long-term performance needs, most AFAs are more upgradable than their HDD brethren. While the concept of an upgradeable storage controller or a scale-out storage system is not new, the advantage that AFAs enjoy is that the media attached to that controller –flash – does not need to change to improve performance. With an AFA the flash is not the bottleneck, the controller CPU and the software are the bottleneck. An upgrade to either of these components improves performance without moving data.
AFAs make primary storage, deduplication and compression practical from a performance standpoint. Server or desktop virtualization makes it feasible from an efficiency gains perspective. Most data centers that we talk to that are using deduplication and compression are seeing a 50 percent or greater efficiency rate and the more virtualized they are the higher that percentage is.
Deduplication and compression are not the only reasons that flash arrays are more efficient. The performance of an AFA also leads to capacity efficiencies. There is no need to use hundreds or even dozens of solid state drives (SSD) to meet the performance requirement. There is also no section of the SSD faster than another section. An AFA allows the IT professional to buy the exact amount of flash capacity they need; it can deliver peak performance at almost 100 percent capacity. Most AFAs should be able to reach 70 percent or more of their potential capacity.
While some AFAs are better at mixed workloads than others are, most of them, thanks to their excess performance, can support more than one workload type. The ability to better support mixed workloads should lead to less total storage systems in the data center. As we discuss in our article “Flash + Object – The Emergence of a Two-Tier Enterprise“, many data centers should be able to consolidate down to two systems; an AFA and a high capacity storage system for unstructured data.
The storage market is going through a temporary adjustment. Organizations are moving to AFAs or flash heavy hybrid systems, but those systems will be serviceable for a much longer period. Vendors and investors need to adjust their thinking and adapt to the new reality of storage.