Archiving to Amazon from your Private Cloud Storage

The only predictable aspect of unstructured data is that it will continue to grow almost unabated. Meeting today’s capacity requirements of unstructured data can largely be handled when organizations deploy private cloud storage, also known as object storage. The more significant challenges are dealing with the rate of growth and the long term retention requirements of unstructured data. To address these challenges may require the use of an external cloud storage provider like Amazon.

Watch our On Demand Webinar "Building A Hybrid Cloud Storage Solution"

The Use Cases For Hybrid Cloud Storage

Hybrid cloud storage is an architecture designed to leverage the best of private cloud storage and public cloud storage. Private cloud storage can be used to store sensitive data that the organization does not want to place on an external storage provider. Private cloud storage could also be used to store data where the latency of the internet connection to the public cloud provider may impact application performance or the user experience. While a complete ROI is beyond the scope of this blog, it is a safe assumption that once an organization’s unstructured data capacity requirement surpasses a few petabytes, the costs of using an external cloud storage provider may exceed the cost of purchasing and managing the storage in house.

But interestingly, organizations don’t need to wait until their data storage requirements are in the PB range to start benefiting from a private cloud storage system. Some technologies allow businesses to implement private cloud storage with as little as 10TB’s but retain the ability to scale-out into the multiple PB range. And as importantly, these systems can be extended into the public cloud to support a wide range of use cases from backup and DR, to user collaboration and file sync and share.

Bursting To The Public Cloud

The first use case, however, for a hybrid cloud storage design is to address the unpredictable nature of unstructured data growth. In some industries, it is not uncommon for data to come in batches. For example, in the energy sector seismic surveys may suddenly introduce PBs of information. In the Film and Entertainment industry, a need to reference an older production may arise while creating a sequel. Even more traditional industries may not have anticipated the data impact of implementing surveillance cameras or voice recordings.

In these cases, having the ability to tap into the near limitless capacity of public cloud storage providers can be invaluable. With this connection, old data can be easily migrated to the public cloud storage provider while the new data is stored locally.

Public Cloud Storage vs. A New Data Center

Another use case for public cloud storage when an organization already has private cloud storage is to limit the growth in capacity within the on premise data center. Public cloud storage is often less expensive than on premise storage if the organization does not have much data to store. As stated above, when the organization reaches multiple petabytes of storage, the private cloud storage solution can be less expensive than “renting” that capacity month after month from the public provider.

Despite the reality that a private cloud solution is less expensive than a public cloud solution, there may come a point when the public cloud solution is more practical. This is because as the private cloud solution scales in capacity, it will require additional data center floor space and of course, more power and cooling. If this scale leads to the need for a new data center, the public cloud solution may be more attractive.

When the organization reaches that point, it may make sense to syphon off older chunks of data to the public cloud storage provider to forestall or even eliminate the need for an additional data center. Establishing a balance between a private and public solution should allow the data center to maximize IT budget.

Hybrid Compatibility Is Key

The key however is hybrid compatibility. The private cloud storage solution should ideally be compatible with a variety of public cloud storage solutions, most notably the nearly ubiquitous Amazon S3. With S3 compatibility built in, connecting a private cloud storage solution to Amazon’s public cloud is relatively easy. Since most public cloud storage solutions provide S3 compatibility, this capability also opens the door to a number of other providers.


Designing and implementing your own cloud storage solution is ideal for organizations that have hundreds of terabytes of capacity to store and see no end in site to growth. But having the ability to offload some of that capacity either temporarily, in the case of bursting, or permanently, in the case of avoiding data center expansion, can have significant value. The key is to make sure that the private cloud storage solution has compatibility with major public cloud storage providers like Amazon.

Companies like Cloudian can enable a business to build a private cloud object storage solution that leverages virtually any commodity hardware. The software can then interface directly into Amazon and other S3 compatible public cloud providers. This flexibility is a key for organizations to effectively manage and mine their unstructured data assets.

Watch On Demand

Watch On Demand

Cloudian is a client of Storage Switzerland

Click Here To Sign Up For Our Newsletter

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , ,
Posted in Article
One comment on “Archiving to Amazon from your Private Cloud Storage
  1. Noam Shendar says:

    Excellent writeup, and matches what we see in the marketplace today. I would add as a footnote that customers looking to do the same with block and file data (as opposed to object) should at least take a look at Zadara Storage, which has private (on premise) storage with the ability to replicate to and from Amazon web Services, Microsoft Azure, and other notable public clouds.

Comments are closed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,249 other followers

Blog Stats
%d bloggers like this: