Reining in Runaway Data Sprawl

Briefing Note: Komprise Data Management Solution

As anyone who has been in IT for a while knows, modern organizations are swamped by what has been called a “tsunami of data.” Various industry sources have indicated that data is more than doubling every two years and project that by 2020 data will have swelled to 35 ZB (zettabytes). These sources also indicate that the majority of this data consists of unstructured data. This unstructured data consists of things like individual user files, rich multimedia files, and machine data; in short any type of data file that is not part of a database application.

Virtualization, dropping disk prices and the Internet of Things (IoT) are among various factors that have helped fuel this tremendous data growth. This has resulted in the proliferation of more and more storage silos of different types within the data center.

Naturally, as data continues to grow unabated, storage sprawl is proliferating at a phenomenal rate to keep pace with this growth. This places ever-increasing demands on IT to manage the sprawl and the data itself.

In response to this situation, organizations are spending an increasing percentage of their IT budgets on the management of this storage and data. The majority of this cost is in data management; storing, searching, protecting and migrating.

Naturally, organizations are looking for ways to not only manage the data sprawl efficiently and reduce costs, but to also find and leverage the value contained in this data for business intelligence, analytics and new use cases among other uses.

Addressing the Challenge

Komprise is a new company that is addressing these challenges with a data management software solution. The solution uses a unique analytics-driven, adaptive automation solution to transparently manage massive amounts of data across an organization’s storage silos, including those in the cloud. Komprise can be deployed without the need for manual migration or changes to existing infrastructure, processes or users. The company claims that data management costs can be reduced by over 70%.

The Komprise solution accomplishes this reduction by creating a control plane that sits to the side of the data path and manages data and storage efficiently while eliminating the need to add new silos to an existing environment. The solution is hardware agnostic so it will work with all storage hardware and the cloud.

Data Management as A Service

Komprise is a cloud solution that operates with an on premises “virtual observer” using a scale out architecture. As more data needs to be managed in more sites, the solution automatically spawns additional “virtual observers” as needed, but they are all managed through one interface and the information they generate is consolidated and managed from one single interface. The analytics backend is delivered as software as a service (SaaS), analyzing the data that the observer collects. The observer runs as a virtual appliance, which is downloaded from the Komprise web site. The company claims that installation and setup is an easy process that can be done in about 20 minutes.

Once this virtual “observer” is installed in a VM in their server environment, it indexes and tags both on-premise and cloud based data. This “virtual observer” does its work by “crawling” the file system and gathering information with a very minimal impact on CPU and I/O operations. The observer works in conjunction with the Komprise Director that runs in the cloud. The Komprise Director provides the intelligence and reporting capabilities that the customer can use to make decisions on their data. Komprise manages both the data and the metadata, but is not in the direct path of any actual hard data.

Data Visibility

The software provides extensive metrics on an organization’s data such as:

  • how fast their data is growing
  • where the data is located
  • the type of storage it’s on
  • what is being used and what is not being used (hot or cold data)
  • when was specific data last accessed
  • where is data growth concentrated
  • when will different storage systems reach full capacity
  • what is efficient and what is not
  • what is protected and what is not

It also provides recommendations on how to improve the placement of their data, the protection of their data, etc. From this information, the customer can then plan on what they want to do with their data and how to leverage it for various purposes.

There is no “observer,” however, for laptops or mobile devices at this time. While not a showstopper, this is a critical void to fill in the product line since so much data exits only on mobile devices and not in the data center. We think that Komprise’s SaaS model is tailor-made for laptop and device management.

Managing Storage Sprawl

Komprise also manages the storage in such a way as to make it appear bottomless, by transparently extending existing storage capacity across storage systems and into the cloud, managing it more efficiently. Komprise does this in a manner reminiscent of early HSM (Hierarchical Storage Management) systems, by transparently moving cold data to other less expensive storage locations and bringing it back to its original location on demand. Part of its analysis is to determine when to move data with the least impact on network bandwidth. From the customer’s viewpoint, the data is still in its original location. All of this is accomplished by the use of symbolic links with the Komprise software acting as a bridge for the symbolic links between different platforms like NFS, SMB, and VMFS.

Bringing it All Together

To bring the analytics and management together Komprise has a WYSIWYG (What You See Is What You Get) type of plan editor through which customers can design, visualize and forecast the impact their plan would have given their particular environment before setting it in motion. It would then continue to track the plan progress as well as adapt over time. Komprise feels this closed-loop adaptive analytics and management system is essential to manage data at scale.

StorageSwiss Take

The Komprise solution has some excellent features. It seems like a step in the right direction and is worth evaluating. Its SaaS deployment model resolves one of the biggest challenges to deploying data management solutions, managing the backend that performs the analytics. It also benefits from the maturing of operating systems so that collection of data can be more accurate and more seamless.

Our only negative comment at this point is its lack of support for an organization’s endpoints. In today’s business environment, data is being uniquely created and stored on laptops and other mobile devices. In order to be a truly comprehensive solution, it will need to extend coverage to these end points as well.

For now, an organization can realize significant gains in data management and reduction in storage costs by leveraging a solution like Komprise. Their SaaS deployment allows for IT to dip their toes in the water with minimal infrastructure investment.

Joseph is an Analyst with Storage Switzerland and an IT veteran with over 35 years of experience in the high tech industries. He has held senior technical positions with several OEMs and VARs; providing technical pre and post sales support as well as designing, implementing and supporting backup, recovery and data protection / encryption solutions along with providing Disaster Recovery planning and testing and data loss risk assessment in distributed computing environments on Unix and Windows platforms.

Tagged with: , , , , , ,
Posted in Briefing Note

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 21,854 other followers

Blog Stats
  • 1,230,921 views
%d bloggers like this: