Applications and users demand unprecedented levels of performance, cyber-threats like ransomware require new levels of security and access control, and organizations are intolerant of any form of downtime or data loss. At the same time, they need to manage data to drive down the cost of storage as well as retain it to meet regulations and provide future corporate value. The result is a storage paradox.
Meeting these conflicting requirements often forces an organization to buy a solution for each use case, which forces compromise and increases exposure. IT needs an end-to-end storage consolidation effort that can provide all of these capabilities in a single system.
What’s Missing For End to End Consolidation to be Possible?
Many of the components to create an end-to-end storage system exist. Flash drives deliver much of the performance that most data centers need. Hard disk storage and cloud storage provide very cost effective, high capacity retention storage. Storage software has become increasingly intelligent in moving data between various tiers of storage.
Missing in action though is integrated security. Certainly, one can add encryption after the fact through software but doing so creates some challenges. First, encrypting all the data that is already there can be time consuming and may require copying it to a new storage system. Second, encryption is not enough. Software needs to analyze and detect potential breach attempts as well as protect against them. Also the requirement to have so many different systems representing each use case opens up the possibility for an excess of administrative accounts, which if compromised threaten to expose the entire organization.
Another missing component in most on-premises storage solutions is true cloud integration. In all fairness, most cloud storage solutions lack any form of an on-premises integration. True cloud integration should be more than just using cloud storage as a DR copy. While a good start, on-premises systems should also leverage the cloud to off-load data from the primary storage system, thus reducing on-premises footprint and costs.
The Problem with Storage System Fragmentation
Today, no storage system has all of the available components and it is necessary to add most of the missing components after the fact. The workaround for these shortcomings is the implementation of multiple storage systems for each use case resulting in storage fragmentation.
Storage fragmentation is a wide spread problem. At a minimum, most data centers have a high performance primary storage system, a mid-range performance primary storage system, a storage system to store backup copies, and a solution to store archive copies.
There is an obvious cost problem with storage system fragmentation as well as a management challenge, as it is necessary to manage each system independently, typically by differing personnel. Long term, each of these systems needs to be upgraded and replaced, forcing IT to face a storage refresh decision on a regular basis.
The Security Problem Fragmentation Creates
At some point after implementing all of these solutions, security becomes a concern. Because the subject of securing the storage system almost always arises after the initial purchase, it becomes necessary to add it to the storage system. Adding security can take several forms. One option is to use self-encrypting drives. Not only will this option require the replacement of the existing drives as well as the migration of data to those drives it also has limited value. Drive level encryption only protects the organization upon physical removal of the drive from the storage system. Typically, this occurs when removing it for repairs or replacement. While connected to its controller, the drive is “wide open”.
Another method for achieving security is to add software to the solution. That software typically creates a new encrypted file system or volume. It is then necessary to move data into the new file system or to the new volume. It is essentially a data migration job and depending on the amount of data to migrate, it can be a time consuming task. Since the encryption software isn’t integrated into the storage software, there can be significant latency in the process of writing and reading data. Essentially, when writing or modifying data as it needs to be re-routed to the encryption component prior to the storage system writing it.
It is important though to realize that organizations need more than just encryption. They need the ability to detect breaches and malware attacks before they occur as well as the ability to stop them when they do. Today this means buying yet another software package that monitors the file system for unusual access activity.
Solving the Storage Paradox
Solving the storage paradox will require a unprecedented level of storage consolidation. This effort is more than just consolidating workloads or even various storage tiers, it is a complete consolidation, isolating down to a single system. It also will need to consolidate protocols, providing File, Block and Object storage capabilities. It will also need to integrate to the cloud, leveraging cloud storage for both disaster recovery and long-term data retention. Finally, it will need to have encryption and breach detection capabilities built into it from day one.
If a single system could accomplish this complete consolidation, then the organization would actually create a much simpler storage architecture that is more cost effective and provides a greater level of security. It also increases security by limiting the amount of super-user accounts needed as well as limiting the number of personnel that need super-user access.
The Architecture Behind End-to-End Consolidation
Architecturally, the ability to provide this level of consolidation already exists for production storage. Storage software today can deliver data movement capabilities automatically moving data between flash and hard disk drives. This basic data management though will need to extend further. It will be necessary to move data to a separate hard disk only system, running the same software or it will be necessary to move the data to the cloud. The ability to move or copy data outside of the original storage system provides both data protection and data management.
Most storage systems can also provide multiple protocols like File, Block and Object storage. This system will need to ensure those protocols are provided without compromising performance or scalability. For protection, the software will need snapshot capabilities as well as the ability to, again, replicate those snapshots to another storage system, with a different snapshot schedule or to the cloud.
The storage software will also need the ability to integrate encryption and auditing capabilities. Encryption should be “always-on” so that IT and the organization’s users never have to worry about data in the clear. If the encryption is consistently applied, it mitigates any performance issues associated with its use upfront and users should never notice its use.
Conclusion
The thought of a single end-to-end storage system may seem like something IT has been chasing for a very long time. Now though, unlike any time in the past, the eco-system is there to sustain such an effort. The combination of fast flash and low cost, high capacity disk plus the cloud, meets both the performance and retention demands of the enterprise. The ease of connecting to remote locations or leveraging the cloud makes protection from small and large disasters more practical.
The missing ingredient was software that knits all these eco-system components together. But now, software can build on what is already available from some primary storage software solutions and add integrated encryption as well as true cloud support.
The end result is that the search for a single end-to-end storage system is no longer an unobtainable dream, it is a reality.
Sponsored by RackTop Systems
Watch On Demand
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 