The industry has been espousing the merits of object storage for years now and cloud providers have been quick to adopt the technology. The enterprise, on the other hand, is more cautious, partly because the object storage alternative, Network Attached Storage (NAS) serves the enterprise well. It is true that like cloud providers, enterprise are experiencing increases in both the capacity and the quantity of unstructured data, but the problem is not as severe as it is for cloud providers. Enterprises make NAS work, despite the advantages of object storage.
Most NAS systems can scale to meet the enterprises overall capacity requirements as well as the number of files they need to store, at least by today’s standards. Capacity and number of files (objects) are two of primary reasons cloud providers select object storage. But there are other reasons to choose object storage beyond capacity and file count. Object storage provides enterprises with a more flexible storage infrastructure that is more scalable, more cost effective and supports more use cases (NAS, iSCSI, File Sync and Share, Hadoop, Object Storage).
There is also the reality that eventually, thanks to the enterprise adoption of Internet of Things (IoT) devices, enterprises will indeed have to face the same capacity and file count challenges that cloud providers are facing today. Based on the enterprise adoption of products like Splunk, CouchBase and Cassandra, they may face those problems sooner rather than later.
Scale-Out NAS Doesn’t Meet Enterprise Demand
Scale-out NAS systems have been available for as long as there have been object storage systems. Like object storage they scale by adding additional servers called nodes that contain processing power and storage capacity. As IT adds each node to the cluster the storage and compute aggregate together to form a single storage system with a single point of management.
Unlike an object storage system, most scale-out storage systems are a tightly coupled cluster of servers. This means each node in the cluster must be very similar (if not identical) in configuration to the other nodes. Also, all of the hardware for the typical scale-out NAS must come from the same vendor. While the all-inclusive approach simplifies implementation, it does cause an expansion problem in the future. As technology advances, either in the form of faster compute or faster storage media, like flash storage, scale-out NAS customers have to create a new cluster for the better storage medium or they have to live with lower performance levels. Essentially, scale-out NAS is only as fast as its slowest component. Ultimately, the organization will manage multiple clusters of scale-out NAS, defeating the purpose of scale-out NAS in the first place.
The Object Advantage over Scale-Out NAS
Object storage systems on the other hand, while also leveraging a cluster of servers, are more loosely coupled nature. This means they can support a wider mixture of server and even storage types in a single cluster. In essence, the object storage system is more scalable since it is able to support a great variety of nodes.
One of the keys to the flexibility is how the object storage systems protect data from loss in the case of a drive or node failure. Object storage system distributes data either through replication or some form of erasure coding. Replication, typically done at an object (file) level, protects data by making sure a use defined set of copies are maintained on specific nodes in the cluster. Erasure coding is parity based. It segments data and then spread across the cluster. Replication provides greater cluster flexibility, but at the expense of additional storage capacity. Erasure coding is more similar to parity-based RAID. It provides greater capacity efficiency but at the loss of some cluster flexibility.
In either case the type of hardware physically uses for the node has little concern to the object storage software. This lack of concern over hardware means the organization can buy high capacity nodes when it is more concerned about capacity and high performance nodes when it needs rapid response times. Most software will allow you to pinpoint what type of objects will go on which type of nodes.
Most object storage software can also pool the types of storage media in the nodes and either present a volume per-class of media (flash, HDD, archive HDD) or it can automatically manage the different media classes placing the most active data on flash and least active on HDD.
Most importantly the lack of concern over the specific hardware also means the organization can purchase off-the-shelf hardware that is very cost effective and best suited for the task. The use of white box servers is only part of the savings, because these systems also use standard server class storage media, both hard disk and solid state disk cost of these devices, are often a factor of 10 less expensive than the same media provided by storage system vendors.
Today’s Use Cases for Object Storage
The value of object storage for the enterprise is not that it can solve one problem particularity well. In most cases object storage is overkill for an individual use case. Again, most enterprises don’t need to store petabytes of images or index the entire internet. For the enterprise, the power of object storage is that it can solve a variety of storage challenges it faces with a single storage system. The combination of these use cases would typically exceed the capabilities of a single NAS and require multiple NAS systems for each use case.
Archive – A true archive is something many organizations lack, even in the enterprise. But it is a storage infrastructure that almost all would benefit from. In most cases, organizations still try to stretch the backup software to fulfill the archive use case. Backup is not archive. An archive project is an ideal first step into object storage. Also there are multiple types of archive (email archive, database archive, sharepoint archive and file archive) that an organization may need to manage. The object storage system can be the backend for all of the archive types. It is critical that the object storage system has the ability to present a NFS or SMB interface to make it easy for the enterprise to move data to the object store.
The paybacks on a well implemented archive strategy are a reduction in both primary storage and data protection architecture expenses. This first step into object storage can easily cost justify the entire object storage investment.
File Sync and Share – Another challenge facing organizations is responding to the Dropbox challenge. Users want to have their data on all of their devices all the time. They also want to be able to share their files with other users or partners when they want to. It is clear that if IT does not provide this type of functionality, users go out and get it themselves. When users do this we call it shadow IT, which puts corporate data at risk of being lost or stolen. There are plenty of File Sync and Share solutions on the market, many of them can leverage either NAS or Object Storage. When IT looks at File Sync and Share on its own, it does not typically justify an object storage system. But it can when IT combines File, Sync and Share with another storage imitative like archive.
Video Surveillance – IP-based wireless cameras are everywhere capturing high resolution images on a constant basis. Many organizations have dozens if not hundreds of the cameras, making the data footprint for storing this data quite large. There are also laws governing how long an organization has to store this data, and the retention time frames are increasing. This means an enterprise has to store the data and make it searchable for years. Organizations are also using video surveillance data for more than just security, some are using facial recognition to track customer activities and behaviors.
The storage infrastructure for most video surveillance systems are often purchased and managed outside of IT’s purview but if IT were to reign the video surveillance data in, something it will probably have to do eventually, it can also be stored on the object storage system, again benefiting from the infrastructure that may be there for archiving or file sync and share.
Analytics – Most enterprises are now either considering or deploying Splunk or Hadoop projects so they can analyze log data and data from IoT devices. Many object storage systems are able to be the storage repository for these projects. Once again the organization can leverage an initial investment in archiving almost “for free” with their Splunk or Hadoop projects.
Cloud Storage – Object storage provides most of the same capabilities as the storage systems that cloud providers use. Implementing one for the organization enables a private cloud storage solution that may be able to reduce dependency on public cloud providers. At the same time some object storage solutions can interface directly with public cloud systems enabling the private system to leverage the public cloud for temporary expansion or long term archiving.
NAS systems have been forced to climb up market and serve as primary storage for virtual machine images, databases and for rare use cases where high performance access to unstructured data is necessary. The problem is most of the unstructured data capacity growth is happening everywhere else. These data require moderate performance, high capacity and extreme cost effectiveness, attributes much more inline with object storage.