Object storage is set to become THE way organizations will store unstructured data. As this transition occurs those same organizations are expecting more from object storage than just a “cheap and deep” way to store information. They expect the system will deliver data as fast as their analytics applications want it. The problem is that in terms of performance most object storage systems are sorely lacking. The reality is the transition to high performance object storage will require more than simply throwing flash at the problem. Underlying object storage software needs to change.
More than Flash
Our entry “The Need for Speed – High Performance Object Storage” shows the decisions to use flash for object storage gets support from improving time to results and increased density. The problem is that “just throwing flash at the problem” will lead to less than desirable outcomes. The key to optimizing a flash investment is making sure the rest of the storage infrastructure does not add the latency that flash removes. This is a particular problem for many object storage systems.
The Object Storage Bottlenecks to Flash Performance
One of the key inhibitors to maximizing flash performance is one of object storage’s biggest advantages; its rich metadata capabilities. The problem is the management of object storage metadata can create significant latency. Organizations looking for flash performance from their object store need to make sure that the system is efficient in metadata management and has the ability to store metadata catalogs on flash or even DRAM.
A second key inhibitor is data protection. Most object stores use either replication or erasure coding to provide protection against a media or node failure. The problem is these schemes consume CPU power, the same CPU power the object storage system uses to run other aspects of its software like metadata management. Vendors need to rethink their protection protocols so they are both network efficient and storage CPU efficient. The key is to limit the re-transmission of redundant data as much as is possible.
A third key inhibitor is lack of hardware flexibility. In the end, object storage software is at the mercy of the processing power of the hardware it runs on. The problem is most object storage systems are tied to a specific hardware relationship and they are slow to adapt new, more powerful processors. To keep up with technological advancements, like flash storage, it is important to select object storage software that can run on a variety of physical server hardware.
High performance object storage makes analytics more valuable by delivering rapid answers to requests. Object storage, powered by flash and support by software optimized for it, enables more frequent analysis across a broader spectrum of data which leads to more insightful and accurate results.