Deep Storage is a tape tier that’s been made accessible to applications and workflows without file system limitations or the complexity of archiving solutions. Using a REST-based interface called “Deep Simple Storage Services” (DS3) Deep Storage can be directly connected to an object storage system, bringing tape’s benefits to workflows that are already using an object-based architecture. It brings these benefits (economics, density, longevity, security and bulk handling) to the environment with a storage format that looks more like disk to applications than legacy tape.
File systems allow applications to interact directly with disk storage, Deep Storage’s DS3 interface will allow them to access tape in the same way, but without the metadata overhead and scalability drawbacks of file systems. With the proper interface hardware and software, like Spectra Logic’s BlackPearl appliance, users can create an open-ended storage tier that combines the ‘forever’ nature of tape with the immediacy of disk and flash storage.
What’s driving need for Deep Storage
Deep Storage is providing a solution for the problem of storing data sets for extended periods of time, often forever. In the past these were compliance-based requirements, but now there are other drivers that may be even more compelling.
More data is being kept to support data analytics than ever before as companies turn to data warehouses and data mining to gain competitive advantage. This ‘more is better’ rationale presumes if we can just look at enough data points we can find something useful. For analyses that are time based this data becomes an historical record of sorts, further encouraging its retention.
Companies are finding new ways to use existing content assets. Movies are converted into new formats, past performances are made into new ‘compilations’ and all recorded contacts are dug up to comprise bios for celebrities that are back in the news. The prospect of being able to resell or ‘monetize’ existing content is expanding the definition of ‘forever data’ and the size of forever data sets.
The Cloud Promise
Cloud-based business models have relied on giving away cloud storage capacity to capture customers. Many of these companies haven’t seen enough of their free customers move to the pay model, as Nirvanix’s recent bankruptcy exemplifies. Deep Storage is their way out by setting up a tape-based tier and using their pricing mechanisms to move some of their customers’ data into it. Deep Storage is ideal if you know the workflow, as cloud businesses do, so they can manage the latency of tape.
Similarly, cloud-based business and cloud operators have to be looking for a way to lower costs. At the end of the day they’re competing with each other, as well as with existing on-site infrastructure solutions. A Deep Storage tier gives them the cost relief they need now and an open-ended solution for keeping the cloud promise into the future. Amazon Glacier is an example of this, from a competitive perspective. Amazon doesn’t give away cloud storage but their customers are the cloud services that do.
With the BlackPearl interface appliance, the benefits of LTFS can also be added to the definition of Deep Storage. As an open-source, file-aware tape format, LTFS enables LTO tapes to be interchanged between platforms that support it. This means Deep Storage data sets can be transported and handled efficiently, in bulk, even between different software applications or different vendors in the workflow.
Storage Swiss Take
DS3 is the ‘missing link’ of sorts in the storage tiering chain. With the advent of web-based data services, REST APIs and object storage there seems, finally, to be a way to make tape more of peer to traditional storage systems. The BlackPearl appliance handles these ‘last mile’ connectivity details of reading and writing data to tape, unlocking its economics and long-term storage efficiency with the concept of Deep Storage.