Content, especially large data objects like videos and pictures, is the storage ‘elephant in the room’ for cloud providers. It’s big and can be overwhelming for their traditional storage infrastructures. But it’s also manageable with the right strategy and the right tools. What’s needed is an understanding of how content is used and a new way to handle, store and access it that addresses the inefficiency inherent in existing storage architectures. Seagate’s Kinetic Storage technology is designed to remove that inefficiency and may represent a paradigm shift as well.
The Content Problem
Content is defined as file-based digital assets that are typically written, stored, read and sometimes deleted – but seldom modified. Examples would be the data that people put into their cloud storage accounts. Unlike document files which are modified but represent a fraction of total capacity, the bulk of what’s filling up the cloud is larger data objects like music, pictures and videos.
Object-based storage systems are gaining in popularity because they handle large content storage environments very well, especially content storage that’s comprised of larger files that have a simple write/read (no modify) access pattern. Instead of using the file and folder hierarchy of a POSIX file systems, object storage uses a key/value methodology to access data. It defines objects by a unique identifier (the key) which it stores in a flat index. Each object is fetched as needed by doing a simple look up of the object ID, which returns a value that’s linked directly to the object.
Using a key/value process would be analogous to using a file tagging system to ‘jump’ to a file instead of scrolling through a directory tree to find it. The benefits for object-based storage systems, where objects instead of files become the ‘atomic’ unit of data handling, are dramatic. Object storage systems greatly reduce metadata and its processing, resulting in an ability to scale performance in conjunction with capacity. But the key/value process has benefits that can extend to the entire storage ‘stack’.
The Storage Stack
A common example of this storage stack has applications sitting on top of a file system and volume manager layer in a host server that talks to a storage controller over one of a number of interfaces, (fibre channel, SCSI, iSCSI, etc.). The controller (or storage server) handles RAID, caching and possibly another volume management layer for the disk drives it supports, to which it’s usually connected via a SAS interface.
With the focus on object storage and internet protocols in cloud storage use cases there’s really no need for the file system, volume management or even the storage controller layers as there is in the traditional model. In fact, if a storage system was designed today for these types of cloud storage use cases it would probably look much different. Such a system would most likely omit these layers altogether and have the storage devices themselves (disk drives, SSDs, etc) communicate directly with applications using the key/value language of object storage. This is what Seagate’s Kinetic Storage does.
Kinetic Open Storage Platform
Seagate has just announced the Kinetic Open Storage Platform, which consists of a new class of hard disk drives that communicate via Ethernet and use the key/value process for accessing data objects. These drives interact directly with applications through Key Value APIs collapsing the traditional storage hardware and software ‘stack’ between the application and its data.
Each Kinetic Storage drive has two Ethernet connections and the intelligence to communicate in a key/value format with Kinetic APIs that are integrated into applications. Seagate has also established a developers’ environment with libraries, developer tools, forums, etc, to help ISVs and manufacturers integrate this technology into their products and promote the adoption of this platform.
By leveraging the key/value architecture that object storage and the web-based IT world is rapidly adopting, Kinetic Storage can remove many of the layers in the storage stack between the application and the storage devices themselves. The result is an immediate improvement in storage TCO (hardware, overhead, power, uptime) plus a reduction in the the complexity of implementing, maintaining and upgrading the entire infrastructure.
Storage Swiss Take
Upgrades in the disk drive industry have historically been all about incremental improvements; fractions of a cent per GB and fractions of Watts. Seagate’s Kinetic Storage technology represents a completely different kind of upgrade. For use cases like cloud storage, it can be an evolution, maybe even a revolution, in storage advancement. Of course, like any revolution, this technology could also be very disruptive, driving some significant improvements across the board in the storage industry.
Reducing complexity, like collapsing the storage stack described above, is one such improvement. Making a system less complex is a fundamental objective of virtually all storage system designs and one that Seagate’s new technology should help achieve. It can reduce the hardware required, make software easier to develop and help address the cost of storing digital content that’s currently threatening the economics of the cloud business model.