Interest in All-Flash storage systems is on the rise from the investment community and it is certainly attracting the interest of IT planners looking to create more scalable database environments and more densely populated virtual infrastructures. All-Flash systems enable scale, and that scale leads to greater return on investment (ROI) from those projects. At the heart of the matter for All-Flash Arrays is where should the “magic” happen…in software or in hardware?
We have seen two different approaches to answering that question. The first comes from companies like Violin and Skyera, who’ve designed innovative, flash specific hardware and then hope to later add software to it. The other approach is to develop flash specific storage software like Pure Storage and then use off the shelf storage hardware.
The starting point may give each an intrinsic advantage that may not be able to be overcome. For example, flash storage manufacturers that focus on hardware design are able to take advantage of the form factor differences of flash. They can use flash as memory modules instead of chips packed into the form factor of the traditional hard disk drive. This should lead to higher performing and more densely packed systems. In other words, more capacity and IOPS in less physical space. Considering that data center floor space and scalable performance are two of the most pressing challenges facing the data center today, these can be very attractive advantages.
The challenge for the hardware focused All-Flash Array manufacturer is that this hardware design takes time and money, more time and money than it does to develop software and use off the shelf hardware from traditional disk system suppliers. This puts them behind the curve when it comes to offering the software features that mainstream enterprises will expect. What is an emerging All-Flash storage hardware vendor to do right now? Again, it takes time to develop software.
Initially, these vendors can and have survived by selling to the performance fringe of the data center that needs high performance and is willing to sacrifice the comforts of robust storage services that the enterprise expects. But, these hardware focused flash vendors are going to want to also capture the mainstream storage market. As stated above, the mainstream needs the performance of flash storage but is not willing to sacrifice the comforts of storage management that they have come to expect from traditional storage arrays.
The mainstream use cases are going to demand capabilities like deduplication and compression, to drive down the All-Flash cost per GB and they will want data protection features like snapshots and replication. These are capabilities that the software focused All-Flash entries are thriving on.
Even the software focused All-Flash Array vendors are lacking a complete offering that the mainstream expects. For example, replication for disaster recovery is commonly missing in action and some OEMs that claim to have deduplication will suggest turning the feature off in high performance environments.
The hardware focused All-Flash Array vendors have a simple solution available to them; use off the shelf storage software (a.k.a software defined storage). SDS brings many of the features that they are lacking with two notable exceptions: deduplication and compression. Deduplication and compression are critical for flash systems to narrow the price gap between them and hard drive based systems. And they need to provide these features with minimal, if any, performance impact. Legacy, off –the-shelf storage software and even modern software defined storage typically lack one or both of these capabilities. All-Flash systems will be expected to support a mixed workload of databases, that are best optimized via compression and virtual environments that are best optimized by deduplication.
One challenge with a ‘use any available off the shelf’ solution is the lack of customization and control. The flash hardware vendor that carefully designed their hardware has to give full control of its presentation to the software vendor that they choose to partner with. The ability to extend and customize the feature set is extremely limited.
The other challenge with these basic off-the-shelf software solutions is that they are “heavy” (i.e. they may do too much). For example, they may include an entire file system architecture or even an object back-end that is simply overkill for a system that needs a simple block presentation of the above features.
The answer for these hardware focused vendors is to integrate with an API set similar to what Permabit offers in its Albireo solution. With this type of solution, vendors can choose to start fast with the complete turn-key offering of Permabit’s VDO solution and then migrate to the fully customizable API driven code of Alberio. In both cases, the hardware focused flash vendor has access to a lightweight, high performance, and scalable storage services offering that includes deduplication, compression and replication.
Hardware focused flash array vendors have taken the time to create systems that treat flash as memory instead of “fast disk.” This allows them to create high performing and densely packed storage systems delivering a very high IOPS per rack unit. The challenge is how quickly they can provide the “rest of the story”; the software that the mainstream enterprises expect. By partnering with companies like Permabit that deliver a lightweight storage services engine, they can get there now and turn what many may have thought to be a disadvantage into an advantage.
Permabit is a client of Storage Switzerland