Harnessing the Data Beast

Briefing Note: Komprise Releases Effective Data Management Solution

As organizations today struggle to contain runaway data sprawl from an ever-increasing avalanche of unstructured data, they are aware that a significant amount (as much as 90 percent in many cases) of data on primary storage is cold, or inactive, data that has not been accessed for a lengthy period of time. However, up to now, managing and migrating inactive data from primary storage is a very time consuming manual task due to the lack of a simple and cost effective means to easily identify cold data. Legacy data archiving and management tools are usually expensive and complex with multiple parts such as agents, special hardware, software and databases that require time and effort to manage.

Additionally, large scale migrations of cold data are often disruptive, creating performance problems due to the management overhead they generate. They are also disruptive for users due to a frequent failure to maintain transparent access to the moved data. This is due to the use of static stub files left in place of the moved data. These stub files can be deleted, orphaned or corrupted. There is also the challenge for these legacy solutions to scale to meet today’s massive data growth. With flat budgets and thin staffing levels, IT requires a new approach to manage this explosive data growth efficiently and effectively.

A Better Way

As we discussed in a previous article, “Reining in Runaway Data Sprawl”, Komprise is a new company that is announcing the immediate availability of its data management software that uses a unique analytics-driven, adaptive automation solution which can transparently manage massive amounts of data across all of an organization’s storage silos whether on premises or in the cloud.

Komprise addresses these challenges with a modern architecture built from the ground up to handle massive scale of data with intelligent automation. The simple to deploy solution is hardware agnostic and consists of two main components, which are a “virtual observer” and a “director”. The director can run as either a cloud service or on-premises. The virtual observer piece runs in a virtual machine (VM) on premises. As more data needs management, you simply add more virtual observers to meet the load. Komprise works seamlessly across any on premises NFS, SMB/CIFS, and object/cloud storage. Komprise also supports NetApp, EMC, AWS, Azure, Google Cloud Storage, Quantum, Spectra Logic and Scality.

Once up and running, Komprise creates a control plane that sits to the side of the active data path and manages data and storage efficiently while eliminating the need to add new silos to an existing environment. The virtual observer works by crawling the file system and tags both on-premises and cloud based data. It does this information gathering with a minimal impact on CPU and I/O operations. Komprise is adaptive, and throttles itself back when the storage is in active use, so that it runs non-intrusively in the background. The virtual observer works in conjunction with the Komprise cloud, which provides the intelligence and reporting capabilities that customers can use to make decisions on their data. Komprise manages both the data and the metadata but is not in the direct path of any hot data.

The analytics engine provides extensive metrics on all data such as data growth rates, data locations and file types, as well as which data users regularly access and which they do not, and which data has protection or not. Komprise also offers recommendations on how to improve data placement. The user can interactively run a variety of “what if” simulations to see the effects of various data policies before implementing them.

Once the user decides on specific data management policy options, Komprise will implement them, automatically managing the migration of cold data to more appropriate storage tiers of the users’ choice, as well as providing for additional data copies sent to different locations as specified by the user for DR or other uses. When it moves data to other tiers, it leaves behind a symbolic link that points back to Komprise itself. Even if the original file moves to other locations several times, the symbolic link points to Komprise, which maintains its own index of the data’s location. If a user deletes a symbolic link, Komprise will detect it and replace the deleted link. It also keeps track of the movement of data to the cloud and can minimize the cost of recalls from the cloud, based on user-defined quantity and cost thresholds. The mapping of links is persisted on the customer’s storage, so Komprise itself is relatively stateless.

The solution also manages data recalls transparently by first placing the recalled file in its own cache. If users repeatedly call for the recovered data, it will automatically move the data back to its original location. From the user’s perspective, it always appears as if the data never left its original location.

The entire operation of the Komprise software uses a single, easy to use graphical interface that allows users to configure and customize it in a variety of ways to meet almost any need. The WYSIWYG plan editor enables users to design, visualize and forecast the impact their plan would have before setting it in motion.

Conclusion

The Komprise solution offers powerful and flexible features that certainly appear to be a step in the right direction and well worth evaluating. It provides true data management and automated, effective archiving in a completely transparent, bi-directional manner.

About Komprise

At Komprise we are passionate about simplifying how businesses store and use their data. Our mission is to help businesses handle the incoming tsunami of data with intelligent automation software that adapts to the customer’s environment and scales across their storage silos, on-premise and cloud, without creating any new ones.

About Joseph Ortiz

Joseph is a Lead Analyst with DSMCS, Inc. and an IT veteran with over 35 years of experience in the high tech industries. He has held senior technical positions with several major OEMs, VARs, and System Integrators, providing them with technical pre and post- sales support for a wide variety of data protection solutions. He also provided numerous technical analyst articles for Storage Switzerland as well as acting as their chief editor for all technical content up to the time Storage Switzerland closed upon their acquisition by StorONE. In the past, he also designed, implemented and supported backup, recovery and encryption solutions in addition to providing Disaster Recovery planning, testing and data loss risk assessments in distributed computing environments on UNIX and Windows platforms for various OEM's, VARs and System Integrators.