Organizations struggling to store and manage an ever rising flood of unstructured data turned to the cloud and object storage in order to contain or reduce costs. But, over time many of them discovered that increasing, recurring costs to store larger and larger amounts of data were in many cases becoming more expensive than an on-premises solution. They also had to deal with compliance, security, data management, data protection and chain of custody concerns. So many organizations started looking for methods to use the same scale out object storage technology being used by Cloud Service Providers (CSP), in order to create and manage their own private or hybrid clouds.
Building a Private/Hybrid Cloud
One very effective scale out software defined storage (SDS) solution that can enable an organization to cost effectively build and manage a private or hybrid cloud is Cloudian’s latest release of their 100% S3 compliant HyperStore 6.0 object storage solution. It is designed to simplify and automate the operational management of cloud storage with new management features that scale to meet the requirements of multi-petabyte, multi-region storage deployments. This lowers management costs while reducing the administrative burden.
This newest release builds on an already strong foundation, which runs on commodity hardware, with features such as peer-to-peer architecture with no single point of failure, multi-tenancy, replication, erasure coding, compression, AES 256 bit encryption, auto tiering to S3 targets, including compatible tape systems like Black Pearl, and QoS (Quality of Service), to name a few. In addition to S3 it also has extensions for NFS and SMB protocols to facilitate interchange between object storage and standard file systems. HyperStore essentially takes all available storage and creates one large storage pool that leverages infrastructure in different data centers regardless of where they are located geographically.
Key New Features in HyperStore 6.0
The new 6.0 release incorporates innovative automation for data storage management, which provides a simple and cost effective means to manage petabyte-scale data volumes. HyperStore 6.0 provides several new features, which are:
- Operations @ scale
New operations console for one click management
Fully automated add/remove nodes on the fly reducing operations management
Non-disruptive automated rolling upgrades and patch installation with no down time
Automatic service recovery
- Durability @ scale
Proactive repair and auto correction, always verified
Dynamic Object Routing for automated failure avoidance
Simple Disaster Recovery with AWS S3 Cross Region replication
- Tuning @ scale
Visual Storage Analytics reports to automatically identify hot spots
Manage clusters with simple, drill down visual management console
Object GPS to locate objects
These new features double the management capacity for storage administrators; provide continuous automated failure resolution for data durability and proactive low cost management for petabyte scale storage. In short, the HyperStore 6.0 release makes it possible for fewer people to manage more storage than ever before.
A Closer Look at HyperStore 6.0 Features
Operations @ Scale
A new management console that provides administrators with an instant 360-degree view into all aspects of the storage system performance and key data. Admins can manage and view all details of every node, wherever it may be located, from a single pane. They can view capacity usage, throughput, transactions, system health and more. Simply clicking on the icon of interest will drill down for full details on that particular object.
All nodes and clusters are displayed in an elegant honeycomb arrangement that lets you see at a glance the status and health of every single node. Simply clicking on any node in the display will drill down to display all information, alerts, health and so on for that particular node and users can take action directly from the interface. This setup makes it easy to see and manage hundreds of nodes. New nodes can be added to a cluster with a single click and configuration of IP address. Once the new node is added, the cluster will automatically rebalance the load.
Other displays allow you to see detailed information on capacity consumption and performance.
The new Capacity Explorer let the administrator see and analyze current data use across the enterprise, which facilitates future planning based on actual demand.
The new displays can also provide an operator with a turnkey view of appliances and internal disks. A unique feature is the ability to click on a malfunctioning drive in the display and make the physical drive actually blink its external LED. This makes it easier for an administrator to quickly locate the malfunctioning drive.
Still another display lets an administrator see the current data distribution scheme and replication operations as well as being able to create and modify storage policies.
Tuning @ scale
Object GPS enables an administrator to locate any object and determine how many copies exist as well as their exact storage location across geographically dispersed data centers. This has always been a challenge for large-scale object storage deployments in the past.
HyperStore 6.0 also simplifies system tuning to optimize the infrastructure for the highest level of system health and data protection. Visual Storage Analytics automatically identify “hot spots” and rebalance workflows for storage I/O protection.
Durability @ scale
By using automated features that constantly evaluate data integrity, even of data at rest, to ensure that storage within the system is always repaired and data is always verified, HyperStore 6.0 extends data durability.
HyperStore’s Proactive Repair function tracks and logs signs of problems with a node, disk, network failure, and incomplete IOs, which is a fast process compared to traditional object storage systems where you have to run a lengthy scan of the entire system to pinpoint the problem. Full system scans can take days during which time user data can be lost if a double failure occurs. HyperStore 6.0 uses its Cassandra database of problem “hints”, as they call them, to quickly pinpoint the problem and rebuild the data in a matter of hours instead of days. It does this proactively, unlike other products that may wait for a read operation or until an admin manually initiates the process.
Smart Redirect for erasure coding and replication schemes enables both a local cache replication for data protection and Amazon S3 cross-region replication for regulatory compliance and disaster recovery while making more hot spares available for erasure coding. The S3 cross-region replication only replicates new PUT commands.
Smart Disk Balancing periodically checks on disk usage and if it detects an imbalance, the software changes the tokens, pointing to a disk with a high utilization rate to one that has a low utilization rate. In the event of a disk failure, new data automatically is routed to newly assigned resources.
Repair-On-Read automatically checks all replicas of an object on a read request. Missing or out-of-date replicas are automatically replaced or updated.
The new enhancements, and features in HyperStore 6.0 clearly make this software defined storage scale out solution one that can give any organization the necessary tools to cost effectively simplify and automate the management of petabyte level private or hybrid clouds while reducing the management burden.
Sponsored by Cloudian