Consolidating storage into a single, or even a few storage systems is a big project. Once the project is complete, IT’s job is not over. The next job for IT is to make sure storage remains consolidated. Temptations to allow for storage sprawl come from everywhere. A new application may demand more performance or bandwidth. There is also the temptation to enable storage sprawl to meet the growing capacity demands of the enterprise. Eventually, even the most disciplined IT personnel will give-in either to appease users or to address real limitations in the consolidated storage system selected.
Why Storage Consolidation Breaks
The two primary reasons that a consolidated storage architecture eventually fragments are the never ending need for increased performance and storage capacity. The need for increased performance typically comes from new workloads that are leveraging a lot of small files. Those small files typically require rapid sequential access, something for which the consolidated storage system may not be designed.
Another reason performance demands cause storage sprawl is the options to increase performance are more numerous than ever. Ten years ago the primary storage media was hard disk drives. Increasing performance was not accomplished by switching storage systems; it was accomplished by adding more drives to the existing storage system. The only time a storage system was replaced was when it couldn’t hold enough drives. Today IT has the option of SAS or NVMe based flash drives which deliver a noticeable performance improvement for many workloads. Each generation of flash array delivers higher and higher performance as storage CPUs become more powerful and less expensive. As a result, when an application owner asks for more performance, the answer is often a new, stand-alone flash based storage array.
The second reason storage consolidation breaks and sprawl occurs is the never ending need for increased capacity. Data is growing at a faster rate than ever and the retention times for that data also continue to increase. Storage systems need to scale to meet that demand but many consolidated storage systems can’t scale to petabyte capacities and even if they can, they can’t efficiently manage the number of files that comes with increased capacity.
Making Storage Consolidation Last
The reality is the hardware behind most storage consolidation solutions simply can’t scale to meet the diverse set of use cases now common in the data center. Organizations need specific hardware for each of those use cases. The concept of storage consolidation is not a lost cause though, it just needs to change and adapt to today’s reality.
IT planners can look at server virtualization and now software defined networking (SDN) as examples of where consolidation is not only working today but is laying a foundation to remain consolidated in the future. The key difference between these two examples and traditional storage consolidation is the true freedom in hardware selection. The customer can pick the server or networking hardware they need to address each specific workload while using the software to manage the server or network environment as if they were a single element.
Storage software needs the same capability. It needs to be efficient enough to derive full performance from high performance hardware like NVMe Flash Arrays, while at the same time supporting high capacity hard disk to meet the needs of protection, retention and compliance. This “storage hypervisor” enables all the disparate hardware to be managed as one unit while enabling workloads to move between them. The problem is many software defined storage solutions don’t provide the required flexibility in terms of hardware selection and use case support.
To learn more about consolidating storage through the use of a storage hypervisor, join Storage Switzerland and StorONE for our on demand webinar “Designing a Storage Consolidation Strategy for Today, the Future and the Cloud.” We discuss how to design a storage consolidation strategy for today, the future and the cloud.