The Best of both worlds – Cloudian HyperStore Appliance

In our prior article, we discussed how enterprise data centers have a need for software defined object storage technology that can leverage commodity hardware to drive down data center costs and to enable businesses to refresh their storage infrastructure with a greater degree of frequency.

However, many data centers also have a need to deploy appliance based solutions to simplify the integration process and to mitigate risk. Cloudian’s HyperStore Appliance solution delivers the best of both worlds – software defined storage that can be implemented on appliance based commodity infrastructure. Let’s take a closer look at Cloudian’s appliance models as well as some of the new features embedded in 5.0 of the HyperStore software platform.

Entry Level To Enterprise Object Storage

The entry-level product, the HSA-1024 is a 1U, 24TB system that is equipped with 32GB of RAM and a 4xGigE NIC. According to Cloudian, this model is ideal for low-throughput workloads like file sync and share.

The mid-tier, HSA-1048 appliance is also a 1U device, it is configured with 48TB’s of capacity,  32GB of RAM and a 4xGigE NIC. With its larger storage footprint, this model is ideal for big content applications like medical records, data center backups and large multimedia libraries.

The HSA-2060 is the top-end appliance. It is a 2U, 60TB system that is optimized for flash and is configured with 64GB of RAM and two NIC devices – 2xGigE and 2x10GigE. Equipped with two SSD drives, this system intelligently places metadata into memory and SSD to help ensure high performance. This model also provides redundancy and high availability features like hot-pluggable disk drives and dual hot-pluggable power supplies. Designed for extremely high I/O bandwidth requirements, Cloudian is positioning this model as their high performance appliance for Entertainment, Energy, Finance and Health Care enterprise environments.

Appliance Powered Simplicity

One of the values to going with an appliance approach is that data center planners can have the confidence to deploy them into their production environments knowing that these systems have been pre-tested and configured to optimally work with the HyperStore software. HyperStore runs on Red Hat or CentOS Linux, however, end-users do not have to be Linux experts to implement or manage the appliance. Instead, the appliance completely shields the Linux operating system from the end-user or administrator and presents an easy to use graphical user interface for system management.

Like the HyperStore software-only offering, Cloudian’s appliances offer native integration with Amazon S3 and REST API protocols. By attaching to Cloudian storage via standard CIFS/NFS, existing applications in the data center don’t have to be recoded to interoperate with private cloud storage or storage resources in the public cloud. This saves organizations time and money and helps to simplify and speed up the implementation of hybrid cloud storage infrastructure.

HyperStore Version 5.0

In addition to their cloud storage appliance announcement, Cloudian HyperStore version 5.0 is a major new software release. Built on a file system enhanced no-SQL database, HyperStore can store vast amounts of unstructured data without the restrictions of being confined to a no-SQL database. This gives HyperStore improved storage scaling and finer control over data availability.

Ring Architecture Scalability

An example of this improved scaling is HyperStore’s virtual node or “vNode” technology. But first, let’s quickly examine the HyperStore architecture. Cloudian storage clouds are implemented by deploying individual nodes comprised of CPUs and disk drives into a logical “ring” architecture. As physical nodes are added, all the resources are aggregated into a common pool of storage and CPU resources across the ring. Rings can start out with as little as one node, however, for redundancy and availability purposes, two or three nodes are typically deployed in an initial implementation. Single nodes can then be added to the ring as needed. Data is dispersed across the ring, via erasure coding or replication to improve availability and to enhance performance.

Virtual Node Performance

Cloudian’s vNode technology enhances data redundancy and availability a step further. The disk resources within a single node can be subdivided into smaller I/O devices (vNodes). This allows for greater I/O parallelism and hence greater storage I/O performance across the ring. Secondly, it enhances availability because if a drive or a node fails, recovery processes can be distributed in parallel across all the drives within the ring to speed up recovery.

Parallel Disk I/O Data Protection

The ability to run disk I/O in parallel across multiple nodes is a critically important feature because as more storage devices are added to the ring, the higher the likelihood that a drive will fail. To compound this problem, disk manufacturers are now coming out with ultra high-density 10TB disk drives. The RAID rebuild times for recovering these drives can easily take 48 hours or longer. Even RAID-6 protected storage systems, which can withstand up to two simultaneous drive failures without incurring data loss, become more vulnerable to data loss as drive rebuild times increase. By leveraging erasure coding in combination with a massively parallel disk I/O architecture, HyperStore can dramatically shrink drive rebuild times and decrease the window of exposure during the repair process.

Erasure Coded Data Redundancy

Erasure coding is a more efficient way to protect data since it doesn’t require configuring dedicated parity drives, like RAID configurations, to provide redundancy. Instead, data or objects can be dispersed across multiple drives within a rack or across multiple racks within the data center and/or racks in offsite data centers. RAID configurations, on the other hand, are bound to an individual RAID-set or drive shelf. This means that as storage grows, there will need to be an increasing number of parity drives to provide data redundancy. This increases costs and due to the issue with drive rebuild times, it doesn’t necessarily decrease the exposure of data loss from drive failures.

It is generally recommended to use erasure coding in a local or regional geography since network latency can be a factor when data is dispersed across extended distances. As a result, there is a need in many enterprise environments for asynchronous replication services when sharing and/or protecting data across wide geographies is a requirement.


Replication has been used to protect data on primary storage systems for decades. It allows businesses to protect against data loss by mirroring or replicating data from their primary storage system to a secondary storage system in the data center or to an array at an offsite facility.

Most object storage systems provide either replication or erasure coding but typically they don’t provide a choice of both. Cloudian’s HyperStore software allows storage managers to not only choose from either erasure coding or replication but they can use both simultaneously. Furthermore, it can be assigned down to a bucket level. So storage managers could configure one “bucket” (a bucket is a virtual container of objects or files) of data capacity with erasure coding and another bucket with replication. This further enhances the ability of an organization to provide multiple service levels as it relates to data availability and offsite protection. One user group may be satisfied with having their data protected via erasure coding across multiple racks in the data center, while another group may require their information to be replicated to a secondary site halfway around the world.

Compared to traditional storage systems which typically can only provide one type of data replication service, asynchronous or synchronous, private cloud storage like Cloudian’s can provide multiple data mirroring services simultaneously. This provides data center environments with more flexibility in terms of the data protection service levels they can apply to their multi-tenant environments. In other words, erasure coding or replication can be defined by individual users or groups of users based on a pre-defined policy or SLA. This is a critical component for providing data protection or DR-as-a-Service capabilities.

Configurable Data Consistency

Cloudian HyperStore also provides the ability to configure the level of data consistency when using replication to protect objects across a storage ring. For example, the default consistency requirement for read and write operations is defined as “quorum”, meaning that a read or write operation must succeed on a quorum (or set number) of replica copies before a success response is returned to the client application.

This enables users to configure how stringent they wish their replication policy to be. For example, for those data objects that are considered mission critical, the replication policy may be to wait until an acknowledgement is received from nodes across multiple data center locations before an acknowledgement is sent back to the application. On the other hand, if performance is deemed more critical, than a correspondingly fewer number of replicas may be configured within a particular quorum.

Storage Node Heterogeneity

Another benefit of Cloudian’s vNode technology is that it enables data centers to intermix node types. In other words, storage nodes deployed into a ring can be of dissimilar size. For example, a 24TB node could be installed alongside a 48TB node and the HyperStore operating system will automatically pool and load balance these resources as they are added to the ring. This gives businesses the flexibility to add capacity and CPU resources as granularly as desired. It also helps to improve efficiencies as the right resources can be added to the ring at the right time.

Compression Your Way

To further improve data center efficiencies, HyperStore 5.0 now offers three different types of data compression technology – lz4, snapp and zlib. Compression can reduce storage and network consumption by up to 40%, while accelerating data replication speeds. With less data to store on disk and less data to move over the network, businesses can get more life out of their existing storage and network investments; further improving their ROI and lowering their TCO.

Encryption Under Cover

With data security breaches becoming more commonplace, as evidenced by the recent theft of millions of account passwords at Google, it is essential for businesses to safeguard their data from the prying eyes of data hackers and unauthorized users. HyperStore 5.0 simplifies the data encryption process by providing transparent key management at the server or node layer. This relieves administrators from the burden of having to manage encryption keys and eliminates the risk of data loss occurring due to lost keys. Furthermore, encryption can be managed very granularly – either at a bucket level or down to an individual object.

Building The Utility

Many organizations, whether a private data center or a public cloud service provider, want to provide storage-as-a-service (SaaS). This generally requires some type of metering and quality-of-service (QoS) capabilities built-into the storage platform. For example, clients that wish to archive or backup their data to low cost cloud storage have very different needs than a user that wants to utilize cloud storage for video streaming. Creating islands of object storage capacity with different performance profiles to meet different service levels would increase management complexity and increase data center costs. At the same time, if all storage resources are fully centralized and aggregated across the same ring, it would be impossible to ensure that the right application workloads are assigned to the right resources.

HyperStore 5.0 solves this problem by providing QoS and metering tools. Storage administrators can set a maximum allowable limit on both storage consumption and I/O, based on the user or a group of users and then chargeback those users on a monthly basis, just like a utility. The CFO could be assigned a high priority privilege (Platinum Service Level) to financial records while an end-user accessing file sync and share data, could be given lower priority access (i.e. Silver). In short, QoS and metering are foundational capabilities for implementing a multi-tenant private cloud storage solution.

Cloud Storage Made Simple

Lastly, the opposing forces of meteoric storage growth and flat or shrinking IT staffs is making storage management simplicity an imperative. To make their private cloud storage very easy to use, Cloudian has redesigned their user GUI to make system management and operations highly intuitive. And as discussed earlier, Linux systems management is done completely under the covers by the HyperStore operating systems so there is no need for storage administrators to have a working knowledge of Linux.


In the late 90’s, commercially available Linux software that could operate on any commodity Intel server platform changed the face of enterprise data center computing. No longer were businesses captive to the huge profit margins exacted by industry behemoths like Sun, HP and IBM. Compute power could be deployed in abundance, very cost-effectively, allowing business applications to attain massive performance improvements through server clustering.

In much the same way, software-define storage and commodity storage infrastructure is ushering in the same monumental cost saving opportunities and continual performance benefits for businesses today. Private cloud storage software, like Cloudian’s HyperStore solution, allows enterprise data centers to implement off-the shelf server and storage hardware to build highly scalable storage infrastructures that can seamlessly grow to meet business data growth demands.

But to make the transition to next generation storage even simpler, Cloudian’s HyperStore Appliance effectively “shrink-wraps” private cloud storage so that businesses can get the best of both worlds – the cost savings and flexibility of highly scalable, low-cost commodity hardware, with the simplicity of rack ready appliances that are supported by a single vendor.

Sponsored by Cloudian

Click Here To Sign Up For Our Newsletter

As a 22 year IT veteran, Colm has worked in a variety of capacities ranging from technical support of critical OLTP environments to consultative sales and marketing for system integrators and manufacturers. His focus in the enterprise storage, backup and disaster recovery solutions space extends from mainframe and distributed computing environments across a wide range of industries.

Tagged with: , , , , , , , , , , ,
Posted in Product Analysis
One comment on “The Best of both worlds – Cloudian HyperStore Appliance
  1. Tim Wessels says:

    Well, Cloudian uses Apache Cassandra and several additional open source projects like Redis and runs everything on RHEL/CentOS. So suggesting that no Linux knowledge is required is a bit of a stretch because even a menu-driven installer and a GUI management console don’t always provide the tools you need to fix something. Cloudian has a number of CLI tools whose use is predicated on some Linux knowledge. The use of vNode “technology” is particular to Apache Cassandra and was explained very well in a recent Cloudian blog. HyperStore is proprietary to Cloudian and was developed to deal with performance issues they were having with Cassandra. That said, Cloudian is a full-featured object storage software platform that is fully AWS S3-compatible along with the extensive feature set you enumerated. It is hard to find a better general purpose object storage software in the market that does all of things that Cloudian can do both for service providers as well as SMB and enterprise customers. Oh, Cloudian also plugs into Citrix CloudPortal Business Manager if you need a self-service portal to access your cloud storage.

Comments are closed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,223 other followers

Blog Stats
%d bloggers like this: