This past week Storage Switzerland and other industry influencers joined Cleversafe, Data Direct Networks, Intel, Nexsan, Quantum, and Scality at the Next Generation Object Storage Summit in Florida. The first order of business was to try to explain what Object Storage is.
What is Object Storage?
Object Storage is a technology where data is stored in self-contained entities called objects. Think of an object as a file. But unlike traditional file systems, this storage method is not dependent on a hierarchical layout of directories and sub-directories. Objects are given unique ID numbers which are managed in a ‘flat’ index reducing significantly the amount of metadata (data about data) needed to store and retrieve a file.
Storage systems that have an object storage foundation can deliver a series of capabilities that should be of interest to data centers that need to store large amounts of files or objects. The classic example is an internet-based business that provides image sharing or file sharing, but these capabilities should appeal to the larger enterprise IT space as well.
Do You Need Object Storage?
The traditional file system or NAS has historically served, and continues to serve, its primary purpose in the data center – high performance file sharing. But as the number of files stored on these systems continues to increase these file systems can become bogged down handling metadata.
An increasing number of data centers are simply adding more NAS heads, not because they are out of capacity but because NAS performance is being impacted by metadata management issues. Object storage does not face this problem since metadata is contained in the object itself, removing the metadata management burden from file system.
A second challenge that the traditional NAS faces is making sure that the data it stores remains valid. Hard drive media can degrade over time and ‘bit rot’ can occur. Object storage, through the use of the unique ID method described above, can provide a continuous protection against this kind of silent corruption. Most of these systems create the unique ID based on the contents of the object and then recalculate it periodically comparing it to the original ID. If its unique ID changes it means the object’s data has been changed, which would indicate corruption if that data was not purposely modified. These object storage systems typically have a method for replacing the corrupted file with a known good copy.
This ability to provide data durability also means that less investment has to be made in data protection. The storage system does not need an elaborate RAID protection algorithm nor do its administrators need to suffer through long RAID rebuild cycles.
A third challenge with a high file count NAS is back up. When file counts number in the hundreds of millions the time it takes any backup software product to walk those files to determine which ones need to be protected can take far more time than the backup window will allow. With an object based storage system it can leverage the unique IDs to make sure that there are always copies of each object available on-site and off-site.
A fourth challenge with legacy NAS is the difficulty of creating an environment that locks down data so that it can’t be changed or so that all iterations of a file can be tracked separately. Object storage solves this problem by once again leveraging the unique ID. The system ensures that once an object’s ID has been created it can’t be changed or in some cases even deleted.
A final challenge is the geographic dispersion of data. The work force today is very mobile and for online services users need data access worldwide. The ability to have data automatically stored in multiple locations based on policy is an important requirement for some data centers. Object storage systems will vary on how they accomplish this but once again they leverage objects IDs to make it happen.
Storage Swiss Take
Object storage is not a new technology; it has been around in various forms for decades. The reason it’s moving back onto the radar screens of data center professionals is that it solves the problems that they are facing today or are getting ready to face, specifically those associated with high file count NAS environments.
Beyond just scalability it also provides key data resilience features that enable companies to maintain data integrity. For organizations that are creating massive amounts of data, data that they can monetize, and where part of the value is in storing that data for a long period of time, object storage can be an attractive option.