Retroactive Deduplication and Compression

In recent months there has been a lot of attention given to how primary storage deduplication is being implemented; should it be implemented selectively, or always on? The problem with this discussion is that it first assumes that the existing storage system will be abandoned, which is seldom the case. Retroactive deduplication is a technology that provides data efficiency (deduplication and compression) to existing storage arrays.

Retroactive Deduplication Completes The Picture

We are seeing an accelerated pace of storage refresh in the data center, caused mostly by the performance promise of All-Flash and Hybrid Flash arrays. IT planners are allured to new start ups that are able to offer flash based systems that leverage deduplication and compression to achieve a better price point than simply adding flash to their legacy systems.

The legacy vendors are left to counter by either deeply discounting their flash offerings, or losing the business. The result is legacy storage vendors like EMC, NetApp, HP, and Dell that don’t offer primary storage deduplication and compression across their entire product line are slowly losing market share to the seemingly endless parade of flash focused, storage startups that do provide integrated storage efficiency.

Retroactive deduplication promises to stem the tide by providing the deduplication and compression features set universally across a vendors storage portfolio without them having to change any of their storage software code. Essentially the retroactive deduplication and compression software installed on an appliance that sits logically in front of the storage system. When designed correctly, this appliance is merely a “bump in the wire”, adding very little latency. Each volume that will see benefit from deduplication can be routed through it.

The Advantages of Retroactive Deduplication

Retroactive deduplication promises the same benefit as the integrated deduplication that is offered in a new system; improved storage efficiency, which is particularly valuable when optimizing premium priced flash storage. But retroactive deduplication delivers this improved storage efficiency, while at the same time providing all the features that a storage administrator has come to count on from their current storage system including snapshots, clones, and replication. While an increasing number of startup arrays now offer some of these features, few are complete and none are as well vetted in production as the legacy vendors are.

Additionally, retroactive deduplication allows for deduplication to be gradually transitioned to on a volume by volume basis. It also allows for deduplication to be leveraged for both flash and hard disk based volumes.

StorageSwiss Take

One vendor that seems to be out in front with retroactive deduplication is Permabit. On a recent Podcast with Permabit’s CEO, Tom Cook, we discussed their SANBlox solution and how it can be retroactively added to any EMC storage system. The solution now available through the EMC select program allows an EMC customer to add deduplication and compression to virtually any EMC array.

EMC’s VNX and VMAX platforms support flash as a tier and can automatically move data between flash and hard disks. These systems of course have the complete compliment of features that storage managers will expect from a storage array and of course an EMC customer already knows how to use these features. EMC, this summer, significantly reduced the pricing of their flash storage options, by as much as 40%. The missing ingredient has been storage efficiency in the form of deduplication and compression. Retroactive deduplication adds that ingredient.

Sponsored by Permabit

Click Here To Sign Up For Our Newsletter

Eight years ago George Crump, founded Storage Switzerland with one simple goal. To educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is and a heavily sought after public speaker. With 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , , , ,
Posted in Blog

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 21,566 other followers

Blog Stats
  • 903,263 views
%d bloggers like this: