PCIe Solid State Disks (SSDs) have become extremely popular in a very short amount of time. They provide uncomplicated access to high performance storage, allowing latency problems to be solved where the application is run – on the server. And, they can do this often without involving anyone else in the organization other than the application owners. However, PCIe SSDs are not without their limitations.
The problem with PCIe SSDs is that they require space in the server and can cause potential cooling problems. They also consume not insignificant amounts of power, consume CPU cycles to gain maximum performance and are not supported on all platforms/operating systems. An alternative is needed that provides similar, uncomplicated server based SSD acceleration without these potential downsides. Putting SSD storage on a Dual In-line Memory Module (DIMM) may be the ideal alternative.
What is an SSD DIMM?
SSDs on a DIMM, like Viking Technologies’ SATADIMM, are NAND flash chips, an interface and a controller (i.e all the components of a typical enterprise SSD) put on a DIMM form-factor PCB. This SSD looks like a DRAM module and fits into any DDR3 DIMM slot available in servers but doesn’t conflict at all with the DRAM already in the servers. That’s because a SATA cable connects the SSD DIMM to any standard motherboard or disk controller; these versions don’t use the memory bus. The DDR3 DIMM sockets are simply used to power and house the SSD, freeing up the PCIe slots for network connection or RAID controllers and the HDD bays for their original function, holding spinning mass storage.
The Advantages SSDs on a DIMM
SSD DIMMs provide several compelling advantages over PCIe SSDs. First are their size. DIMM slots are available in some quantity in almost every server or workstation. PCIe, especially the full sized slots that many PCIe SSDs require, are increasingly rare in smaller 1U servers. Even if PCIe SSD makes sense for the server or storage system, it simply may not physically fit into that server or storage system.
Second, at 500GB per SSD DIMM, storage can be added more incrementally, another advantage related to its small size. This flexibility can make them more economical than PCIe cards which typically have a 1-2TB capacity.
Third, an SSD DIMM leverages a standard motherboard or add-on controller. To those controllers the device looks like another hard drive. Unlike most PCIe SSDs that require an operating system-level driver an SSD DIMM will work in any platform that has a SATA controller. In contrast, most PCIe devices have Windows drivers, a few have Linux drivers, a few less have VMware drivers and very few have Macintosh support.
These platforms that are behind the curve in support are also the environments that could most benefit from SSD performance. VMware workloads are very random nature making them ideal for SSD. Linux has become a ‘go to’ operating system for host database applications and SSD technology is actively accepted by the Mac community (Apple is now the number one consumer of Flash NAND). Putting SSDs on a DIMM gives each of these platforms a new and highly flexible option for increased performance.
The support of standard SATA controllers also means that the feature set of these controllers is available to SSD DIMMs. The most immediate benefit is the ability to boot from an SSD that’s on the DIMM. Most PCIe SSDs cannot support this, meaning that the server must still have a mechanical hard drive to contain the operating system. This consumes a drive slot, slows down boot time, may impact virtual memory performance and complicates storage management by requiring two devices where one may have sufficed.
Beyond just boot, there is also the ability to leverage advanced features that are increasingly common on SATA based controllers, things like advanced RAID protection, snapshots and better cache management. These capabilities are often included at no additional charge with the controller and are, of course, a high value to the storage manager.
To get these features into a PCIe-based SSD the PCIe vendor is going to have to build them in. This requires either an internal development effort or integration of a third party solution which may add latency and negatively impact the PCIe performance.
More Server Friendly
Leveraging the DIMM slot also makes this SSD implementation more server-friendly. In a server, especially a smaller 1U model, PCIe cards can restrict airflow, bringing a host of heat-related problems. One solution is to have cooling fans spin at a higher RPM so more air can be pulled through the server. This results in more power being consumed and more noise, since faster fan speeds make for louder servers.
A RAM Alternative
DRAM is still expensive, while it has decreased in price it has not decreased at the rate that Flash based SSD have. So many servers have available DIMM slots (depending upon the application); it’s simply too expensive to fill them up with maximum capacity DRAM modules. DIMM-based SSD technology should be looked at as a serious alternative to expensive high capacity DRAM. Since approximately 500GB can be stored in a single SSD DIMM it provides far more capacity per DIMM slot than DRAM can. The operating system can then use this storage as a paging area for RAM operations.
In the past, virtual memory and paging to disk were considered a performance problem, now memory would be paged to a solid state storage area. SSD DIMMs and any flash storage is not quite as fast as DRAM but is still significantly faster than a mechanical hard drive. A growing amount of test data shows no noticeable performance loss when paging to SSD.
The SSD Storage Appliance
Many SSD vendors are trying to leverage PCIe SSD technology to create a shared storage SSD appliance to compliment their server based solutions. The problem with this approach is the space and density required. Standard off the shelf servers have a limited amount of space allocated to PCIe expansion. Conventional server design does not factor in the need for a bank of PCIe slots. Even a custom design is limited to how many slots can be allocated to PCIe.
Comparatively, SATA-connected SSD DIMMs only need the physical footprint of a DIMM slot. In that slot Viking’s SATADIMM, for example, can house up to 500GB. Leveraging available motherboards that are primarily a bank of DIMMs could deliver an extremely large amount of SSD storage in a 1U server.
By integrating the storage connectivity of choice a shared SSD DIMM storage array could be configured that sets a new bar in SSD density, performance and cost per GB. As a proof point Viking Technologies demonstrated just such an array with 32TB of capacity in a 1U chassis, using their SATADIMM storage devices.
While PCIe SSD garners the lion’s share of attention it is not without its limitations. DIMM-based SSD technology can provide much of the appeal of PCIe SSD without many of these downsides. The key issue is whether PCIe SSD will even fit into the servers that need solid state acceleration. If so will they supported that OS and have the feature set that storage managers have come to expect? SSD DIMMs meets all of these challenges and may be the alternative to consider.
Viking Technologies is a client of Storage Switzerland