VMware fundamentally changed server deployment. Prior to VMware the best practice was “one-server, one-app” which, as Intel processors continued to increase in power, led to a massive waste of compute resources. VMware allowed applications to be safely stacked on those servers so that applications could make optimal use of compute power. The move to VMware also provided freedom from buying specific hardware. While hardware was still important in terms of CPU and memory, the actual vendor of the hardware was far less critical. IT could install it literally on almost any hardware and VMware would run as well as the capabilities of the hardware allowed.
Software defined storage (SDS) was supposed to provide similar benefits to the storage infrastructure, but so far it has provided very few.
Comparing SDS and VMware
During the last decade, most advancements in CPU power have been through the increase in number of CPU cores. VMware leveraged those cores and derived more efficiency from the existing server hardware by allowing IT administrators to stack multiple applications on to the same system. In other words, it put all those cores to work. Other than understanding peak behaviors, they didn’t need to be overly concerned about the types of workloads. VMware can easily support VMs performing random IO and other VMs performing sequential IO all on the same physical server.
Storage systems don’t attain the same level of optimization. One seldom sees the raw IO potential of the internal storage devices once they are in a storage system. Adding more CPU horsepower to the system will reach a point of diminishing returns. The primary problem is the storage software itself. The overhead required by the storage software adds significantly to latency and decreases performance. To work around their overhead, SDS vendors will specify specific server types with more processing power, a minimum number of drives and faster drives. While performance improves, the gap between what the system provides and its raw potential actually widens. It becomes worse than a case of diminishing returns; it becomes a case of increasing losses.
The primary culprit is the efficiency of the software itself. Storage software for the most part is using the same algorithms for reading, writing and protecting data that it did years ago. Also in their haste to get to market, many vendors leverage open source libraries to deliver certain critical functions to their software. Most of these libraries are generalized on purpose to be used across the broadest of use cases instead of being optimized.
SDS’ Unique Hardware Lock-in
The lack of efficiently in using the compute, memory and storage resources provided to it leads many SDS vendors to violate their own mantra “no hardware lock-in”. Since SDS vendors want their software to achieve a certain level of performance and reliability, they end up being very specific as to what hardware they allow the customer to use with their software. In many cases, the very specific nature of the requirement creates complexity and increases cost, leading customers to tell vendors to provide the hardware and software together as a bundle. The result is that SDS vendors end up looking very much like the legacy proprietary solutions they want to replace.
Comparing Track Records
VMware has an impressive track record of success. The primary reason is that it worked. Early adopters could test the solution on whatever server hardware they had laying around and were, in most cases, stunned at how well it worked. VMware effectively breathed new life into retired systems.
Most software-defined solutions can’t be installed on old hardware lying around, even during the initial testing phase. Certainly, no software is going to deliver all-flash performance from a hard disk based server but it should deliver the maximum performance available. On the other hand, it should allow the customer to install a few flash drives in the server and see how the system performs then. Even if the SDS solution supports using an old server with a few new flash drives the results are often less than impressive.
How Holistic SDS Changes Things
In a recent paper “What is Holistic SDS?” Storage Switzerland introduced the concept of Holistic SDS which is storage software written from scratch with new algorithms and design optimized for multi-core servers. Holistic SDS allows IT administrators to test the solution in the same way they tested VMware. Implement it on an old server, install a few SDS drives and see how it works. What they should find is that the old server has new life breathed in to it even while using older CPUs and just a few commodity flash drives. With Holistic SDS, even these modest configurations should deliver hundreds of thousands of IOPS.
Holistic SDS is the next generation of SDS. It is also likely the first generation to be broadly adopted because it emulates VMware’s path to acceptance. The key is very efficient use of existing hardware allowing IT to run the software on almost any system with any configuration of storage. It allows the mixing of sequential and random workloads and is applicable for multiple use cases; production storage, backup storage and even archive storage.
To learn more about Holistic SDS register for our live webinar “Why SDS is Broken – and How to Fix it”.