How Cloud can enable the All-Flash Data Center

It’s been said that, given the choice, every data center would choose to be all-flash. This is because access to near limitless performance not only increases application scale it also simplifies many day-to-day storage management tasks. This potential has led all-flash to be one of the hottest trends in storage. But most IT planners realize that their organization can’t afford to go all-flash. The cloud, another hot storage trend, could be the key to enabling the all-flash data center by making sure it is used optimally.

The premium cost of flash has lead organizations to treat all-flash arrays as a point solution to solve specific performance problems. Simply put, IT professionals know that not all data needs to be on flash storage all the time. But this is a data management problem, not an all-flash problem. The question should be asked: Does this data even need to be in the data center? What if it could be removed, so that all that was left was data that could justify residing on flash storage?

The 90% Problem

Most data centers report that less than 10% of their data is truly active. Many data centers could easily afford to put 10% of their data on an all-flash array; the question is where to put the other 90%. That alternative storage needs to be available enough that when this non-active data is needed again it can be easily accessed, but out of the way otherwise.

The typical solution to the 90% problem is to ignore it and treat all data the same. This of course prevents the broad based use of all-flash storage and relegates it once again to a point solution. Another option is to move data either manually or automatically to hard disk based storage in the data center that works alongside of the all-flash array.

This technique has its own set of challenges. First, can the data movement be unified across a variety of storage vendors? Most all-flash arrays don’t have this capability built in, so the hard disk system will be different than the all-flash system. Second, it really doesn’t solve the data center problem. Hard disk systems still need to be purchased, powered, cooled and maintained. Also utilization of capacity on these systems is rarely even close to 100%. As a result the organization is still paying for unused capacity. Most importantly they still continue to take up valuable data center floor space.

The Cloud Solution

The cloud could be the ideal solution to the 90% problem. It can be purchased as needed and is always at 100% utilization. It requires a limited upfront purchase and does not require that the organization power, cool or maintain that storage. Finally it takes up zero data center floor space. When the organization commits to an all-flash, hybrid cloud solution, they may free up a lot of data center floor space.

Avere 3569 Welcome BannerThe challenges of course with cloud storage are the uncertainty of performance, the cost of an ongoing storage bill and of course concerns about data security. There is also the concern about data movement, how to get the right data in and out of the cloud as quickly as possible.

The Hybrid Cloud

To help ensure that application performance is not impacted by the latency of moving data to and from the cloud, data centers can take a hybrid cloud storage approach; in other words, co-locating business data i.e. maintaining an onsite copy for local access while replicating data into the cloud for backup/archival and DR purposes.

The hybrid cloud should also be able to mitigate concerns about on-going cloud storage costs. First, in the cloud the organization only pays for the exact amount of storage it is currently using. There is no unused capacity in this model. Second, many cloud providers are offering tiers of storage within their cloud. The hybrid cloud solution should be able to leverage these tiers, and as data ages move it to less expensive cloud storage tiers, while still making that data seamlessly available to applications and users.

The hybrid cloud model has been available for several years and is used extensively for backup and archiving. But the typical connectivity between on premises devices and cloud is more of a gateway. The on-site appliance usually is not designed to meet the performance demands of active applications and users. To enable the all-flash data center the performance capabilities of the on-site device need to change from one of a gateway to more of a storage appliance designed for performance.

The Hybrid Cloud Appliance

Instead of a gateway approach, the on-site device in the hybrid model needs to be an all-flash primary storage appliance. It needs to be designed to provide high performance as well as offer seamless interaction with data whether it is locally available on the appliance or solely in the cloud. At the same time, the appliance should not try to become a “system” offering data protection capabilities like replication.

It also makes sense for this appliance to provide the actual on-site flash capacity. Doing so will reduce cost and complexity and allow the data center to rapidly move to an all-flash reality.

Companies like Avere have taken this approach and it has allowed them to focus on important capabilities such as creating a high performance global file system that meets the demands of tier 1 applications, while providing seamless access to data regardless of location. It also allows them to focus on developing accurate data placement algorithms so that cache misses are kept at a minimum.

For applications that need to also be hybrid (run on local compute and in the cloud at different times), solutions like Avere’s Virtual FXT Edge Filers are a good fit. The Amazon Web Services (AWS) compute cloud has a storage component called Elastic Block Storage (EBS). It consists of flash storage and hard disk drives storage. Virtual FXTs can automate the movement of data between those tiers for cloud-based applications.

Cloud Security

There are two security concerns when using cloud storage. The first is making sure that the data is readable only by authenticated users and applications. This can be done by leveraging AES-256 encryption, which many solutions have. But the enterprise needs something more: full key ownership. The hybrid cloud appliance is an ideal location to manage the encryption keys and ensure that no one “cloud-side” can read the data. For example, the Avere architecture supports the KMIP protocol for managing and storing encryption keys. The cluster uses this standard interface to work with encryption key management solutions that have been explicitly built to manage keys.

The second concern is cloud storage lock-in. It is easy to get data into the cloud, it is more difficult to move that data out of the cloud. This is a problem because there may come a time when the organization wants to move all or a portion of its data to another cloud provider. This can be to derive better pricing or to meet a data sovereignty concern. The hybrid cloud appliance should be able to support multiple cloud backends and seamlessly, thanks to the global file system, switch between them.

Backup and Disaster Recovery Built in

One of the distinct advantages of hybrid cloud storage is the fact that organizations can eliminate the significant capital and operational costs of building and maintaining a dedicated secondary or tertiary data center for safely off-setting their critical business data. For the most part, the hybrid cloud provides backup and disaster recovery built in.

In the event of a data center outage, another appliance can be initiated at an alternate site and begin accessing data instantly while it is being restored.


If data centers used all-flash just for their truly active data, an all-flash data center is viable today. The cloud solves the problem of where to put the remaining 90% of data, that can now be deleted from local storage, but could be needed in the future. To make this connection requires an appliance that can not only provide high performance local storage but intelligent data movement to and from the cloud. Companies like Avere Systems are providing solutions like their FXT Edge Filer Series to pave the path to the all-flash data center.

Sponsored by Avere Systems

Click Here To Sign Up For Our Newsletter

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , , , , ,
Posted in Article
One comment on “How Cloud can enable the All-Flash Data Center
  1. Joseph J. Kobylecky says:

    For someone just starting to understand the items that make up considerations for the type of Clouds for storage, I would like to thank you for presenting the material clearly.

Comments are closed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,031 other followers

Blog Stats
%d bloggers like this: