olid state drive (SSD) solutions using flash are becoming the ‘go to’ options for addressing storage performance bottlenecks. And within that technology category, PCIe based SSDs could represent the state of the art. By locating the flash in the server they can eliminate the latency of the storage network but can also re-introduce the storage silos that storage networking was designed to eliminate. This begs the question: “Is there a way to leverage server-side PCIe SSDs without breaking the storage network?”
The Advantages of PCIe SSD
When storage systems were all hard-drive based the latency of the network and storage protocol stack were negligible, relative to the speed of electromechanical drives. The time required to spin disk platters and move drive heads overshadowed any of these potential bottlenecks. Flash storage, with its memory-speed response time, has almost no latency and as a result can expose the SAN and even the drive interface along with various other stacks as potential bottlenecks.
This exposure has led to the creation of PCIe SSDs, which are installed directly into the server where the demand for storage performance originates. In server-attached flash, capacity is local to the application that needs the performance boost. No longer does the application have to negotiate the interface protocol stack or take a trip across the network to access data; the CPU has essentially direct access to the data. The result is near-zero latency performance and very wide, PCIe-driven bandwidth. For the application owner the payoff comes in massive performance gains that can lead to greater scale.
How PCIe SSDs Break the Storage Network
Despite these nearly obvious advantages the mainstream data center, other than in a few isolated use cases, has not been adopting PCIe SSD as quickly as expected. Most of the adoption has been by Web 2.0 and hyperscale environments where the software code is owned by the organization allowing them to program around the shortcomings of PCIe SSDs.
The first problem mainstream data centers have with PCIe flash is that it recreates silos of storage on a per-server basis, similar to those caused by direct-attached storage based on hard disk drive (HDD) technology. In those scenarios storage was captive to a single server, which meant it could not be used efficiently across the data center. Since flash-based storage is more expensive than HDD storage the pain inflicted by this lack of efficiency is worse.
This ‘siloed reality’ also leads to data protection and availability challenges. Certainly most PCIe flash solutions can be mirrored for redundancy but this further increases the pain of inefficiency. While technically many can be put into a RAID group the availability of PCIe slots in most servers won’t support enough cards. Also, isolating these cards in a single server creates a potential for loss of data or availability if that physical server has to reboot or sustains a hardware failure.
In addition to these hardware challenges, the software to support PCIe SSDs has been fairly limited as well. Most PCIe vendors have been focused on delivering the core drivers that make their cards recognizable by operating systems and environments. Generally, they haven’t made the investments required to enable their software to work around these efficiency and redundancy shortcomings.
Despite these downsides the adoption rate of PCIe SSDs is slowly improving, even in the mainstream data center. This is because the potential performance benefits are great enough to outweigh the shortcomings. But for accelerated, widespread adoption in mainstream data centers PCIe vendors need to invest in software solutions that will circumvent these shortcomings without impacting the raw performance advantages of this technology. Companies like Virident are out in front of the problem, delivering software solutions to address the potential problems of server attached high performance storage.
Software – The Key to Mending the Broken Fence
As stated above, despite their disadvantages, PCIe SSDs are seeing increased acceptance. To broaden the appeal PCIe vendors need to invest in software designed to work natively with their hardware and eliminate these shortcomings. This however, cannot be a one-size-fits-all solution since different data centers will have different requirements.
PCIe SSD with Caching
For most data centers this first requirement is for the PCIe solution to compliment the shared storage systems that are already installed in the environment. In this case data centers are looking for PCIe SSD to act as a cache to shared hard drive based systems. These implementations extend the useful life of both the shared storage system and the storage area network (SAN).
This caching or ‘storage augmentation’ role of PCIe SSD may also be a long-term strategy. Even if an organization decides to invest in flash on the SAN they still can take advantage of the local performance and reduction in network latency that PCIe SSDs provide.
In either case it is important for PCIe SSD suppliers to provide caching solutions that can support a variety of platforms that the data center may require like Virident’s vCache software does. Caching can be an easy first step into flash based storage because it automatically provides acceleration of the most active data sets. In some specific cases with a read-only or write-through cache mode there is little need for high availability (HA) or even mirroring, so the entire cache capacity can be allocated to the acceleration of data.
PCIe SSD with HA
There are some data centers, or applications within those data centers, where the latency from a cache miss can’t be tolerated. They truly need server-side PCIe performance but also the reliability of a shared solution. For these environments PCIe SSD vendors need to develop software that can provide a simple, HA capability without having to create a full-fledged shared storage infrastructure. They may only have a few applications that need this level of availability so a simple server-to-server mirror is ideal.
For these use cases an HA solution like Virident’s vHA which provides high speed synchronous mirroring between servers, can be ideal. In this design the PCIe SSD storage is replicated directly to another server for redundancy. While adding a little more latency than an internally mirrored solution it’s not enough to impact the overwhelming majority of applications and for this small amount of latency, data availability increases significantly. A solution like vHA is a very cost effective way to reliably accelerate mission-critical, performance-critical applications.
Server Networked PCIe SSD
Finally, there are applications that require a widespread deployment of flash storage but still need the performance benefit of PCIe SSD. This is especially true for clustered environments like Oracle and VMware where shared access, high performance and high availability are all critical.
For these environments a “server network” is an ideal way to meet all three demands. While similar to a SAN a server network can be dedicated to flash storage exclusively. This contrasts to a storage area network where flash must share bandwidth with much slower devices like traditional hard disk storage and even tape. This flash-only focus means a simpler network with more available bandwidth and less blocking, which increases utilization of compute and flash performance.
The PCIe cards in each server are aggregated through software like Virident’s vShare into a globally available PCIe SSD pool of storage. This pool can then provide a high performance and redundant shared storage area for clustered applications and environments.
Conclusion
Server-side flash, and particularly PCIe SSDs, can be ideal for reducing storage latency and improving application performance. But locking flash capacity into the application server can reintroduce some storage issues around efficiency and availability that shared storage had long ago resolved.
Value added software that enables flash pooling, caching and HA among servers can help organizations address these issues. It can also allow PCIe SSDs to move from being a ‘one-trick pony’ for the performance fringe to becoming a full-fledged data center solution that can solve a wide variety of storage performance and availability challenges.
Virident is a client of Storage Switzerland
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 