Recently the David Wright, CEO of solid fire wrote an excellent blog responding to a column I wrote comparing the benefits of scale-up and scale-out All-Flash storage systems. However, his blog might lead one to believe that I think scale-out storage has no role to play the data center. Let me be clear, I believe that scale-out storage systems do indeed have a role to play in both enterprise and cloud data centers.
The point of that original column was to bring some equality to the discussion. I think too much attention is being paid to scale-out technology and not enough to the capabilities of scale-up technology. The other important aspect that seems to have been missed is that I was focusing on the storage needs of the medium to large-size enterprises, not cloud service providers or webscale/hyperscale data centers. In this column, I will discuss where we see potential advantages for scale-out storage systems.
Scale-Out Storage for Unstructured Data
Clearly there is an opportunity to use scale out storage for unstructured data. As we have spoken about on several webinars this year unstructured data is growing at an alarming pace. This is fueled not only by users creating more of their own data but also by machines creating data (known as “the Internet of things”). While there are some situations like analytics where performance is all important, the unstructured data storage use case is typically more capacity focused. In these environments the demand is for PBs of data and scale-out storage makes a lot of sense. There are two competing thoughts on how to handle unstructured data too; Scale-Out NAS vs. Object Storage. For details about that comparison see our article “Scale-out NAS vs. Object Storage”
Performance Focused Scale-Out Storage
Does scale-out storage belong in the performance context? As stated in the original article I believe that it does and SolidFire is an example of a storage system that provides this scalable performance very well. But the number of data centers that could actually take advantage of a scalable performance model is limited. Not surprisingly SolidFire is focused primarily on cloud providers and very large enterprises; environments that can justify a scalable performance storage system. These environments also need features like multi-tenancy, quality of service and customizable interfaces, all of which SolidFire provides.
Is Scale-Out Performance Storage For You?
Whether scale-out storage is the best choice for a particular environment depends primarily on how fast or how unpredictable your performance and capacity needs will change. As Wright correctly pointed out, it really comes down to whether you will need to scale performance or capacity beyond what a scale-up storage system can do over a given time; five years as an example.
Consider a scale-up storage system that can deliver 400,000 IOPS and 25TBs of raw capacity. If you are confident that your performance demands in total won’t exceed 400k IOPS over that time then you probably won’t need scale-out storage from a performance perspective.
Capacity might be the harder variable to isolate. An All-Flash Array will likely bundle deduplication and compression and as a result can deliver more than 25TBs of capacity. But exactly how much more will be largely dependent on the type of data you store. For example, in a virtual infrastructure (server or desktop) the data reduction rate can be quite high. But in a database environment it will be more modest and if you’re storing images or video the impact of these data reduction technologies will likely be nonexistent.
Most data centers are looking to All-Flash Arrays to help with the performance of their database and virtualized environments, so it’s reasonable to expect at least a doubling of that capacity. Pure Storage, via their real-time storage efficiency ticker reports about a 6.3 to 1 average data reduction. Using that rate, 25TB of raw capacity would seem like 150TBs; more than enough for many data centers. Combining these two results makes scale-up storage compelling, especially if it’s less expensive.
But what if you are in a second group of customers that can’t exactly down their performance and capacity needs over the next five years? Then considering both scale-up and scale-out won’t hurt, especially if you can find a scale-out solution that is in the same price range as a scale-up solution. The key is to clearly understand the starting costs and networking requirements of the similarly priced scale-out solution.
However, if you’re in a third group of customers, those that are experiencing rapid growth and know they will exceed both the performance and capacity potential of a scale-up system, then yes, you should consider scale-out storage. The obvious target for these types of solutions are the webscale/hyperscale data centers, but also some large enterprises will experience these same rapid growth challenges.
More Than Performance and Capacity
It is also important to remember that there are reasons other than performance and capacity to buy a storage system. Good examples are some of the features I mentioned above, like quality of service, multi-tenancy and programmability, may be as important in your situation. Those would need to be factored into the decision-making process.
Every data center is different. The methods used for managing increased performance and capacity demands will vary widely. Both Scale-Up and Scale-Out storage systems have a role to play in meeting those demands. Unless they are pushing their applications to the cloud, most medium-sized to large data centers will be well served by scale-up systems. For the cloud provider and very large enterprise, scale-out will make more sense.