If primary data is growing, the copies made of that data are exploding, with a growth rate expected to be 10X that of primary data. These copies are most often used for data protection, test/development, compliance, archive, and analytics. Fortunately, solutions to reduce the number of data copies needed and to manage those copies are appearing on the market. In fact, Storage Switzerland expects 2015 to be a watershed year for copy data solutions. With the recently released second version of their product, Catalogic is quickly asserting itself as a company to watch in the copy data space.
Catalogic recently announced the general availability of ECX version 2.0. This version manages the full lifecycle of copy data, from creation through archiving, as well as the ability to manage, orchestrate and analyze copy data. Most importantly it adds copy data functionality to existing storage systems instead of forcing an organization to rip-and-replace.
Snapshots: The Foundation of Copy Data Management
The foundational technology to managing copy data exists on almost every storage system today: snapshots. Most storage systems also have the ability to replicate those snapshots to a second or third system. This protects against the loss of snapshots due to a failure of the primary storage system. Finally, most storage systems also have the ability to schedule when those snapshots are taken.
What is Missing From Snapshots?
The problem is that most snapshot technology ends at the point of taking the snapshot and potentially replicating it. Storage systems typically provide limited ability to search a global snapshot pool for specific versions of data. Instead, they force the storage manager to find the needle in the haystack. They also lack the ability to automate the deployment of snapshot data for other uses, like mounting a snapshot for data protection, creating a writeable clone for test/dev or delivering snapshot data to an analytics process. If the storage manager is forward thinking enough to want to use snapshots in this manner they have to manually present snapshot copies to these processes.
Completing The Picture Without Recreating The Foundation
Some copy data solutions create their own snapshots and data capture methods, instead of leveraging what’s already in place. This means learning a new snapshot creation process and trusting it to provide data integrity. Why not extend the current snapshot technology that’s already being used and trusted, instead of replacing it?
With version 2.0 of ECX, Catalogic completes the copy data management picture without re-laying the foundation. It adds a catalogue that can be searched to find data across snapshots, and provides automation so that copy data can be automatically presented to backup, test/dev, compliance, archive and analytics tasks.
Installation requires downloading and installing a virtual machine (VM) and then registering your storage systems with the software. The catalog can read snapshots that are already in place and is ready for searching and other actions within minutes.
This iteration of ECX is focused on providing copy data management solutions to NetApp customers. This year they will add support for other major storage system manufacturers’ solutions, as well as a few startups.
As we discussed in our article, “What is Copy Data?”, Copy Data Management is an important capability that many data centers will want. We expect to see a flood of copy data management solutions come to market in the next year to 18 months.
A key consideration will be how distributive these solutions are to the existing infrastructure. Solutions that require agents installed in applications, drivers installed at the operating system or that replace the storage system altogether, will face an uphill challenge. The path of least resistance may be to augment what is already in place, as Catalogic has done, building on the storage system’s capabilities instead of replacing them.