When flash storage first came to market it exposed the weakness in storage designs, hidden by the latency of hard disk media. Now, NVMe promises to expose weaknesses in the designs of the first generation of all-flash arrays. One of the significant areas of concern is the storage controller. If the storage controller can’t handle increased IO, then the performance potential of NVMe-based flash storage will mostly go unrealized.
The Storage Controller Problem
The biggest challenge in reaching the full potential of the NVMe-based flash system is all IO has to flow through the CPUs inside the storage system. In a busy environment, the CPU becomes a bottleneck. Scale-out systems, by themselves, are not the solution since most scale-out systems have a primary node that is responsible for IO distribution. It becomes a bottleneck because the IOs are still not processed in parallel.
The other problem is that the servers are attaching to this shared storage through traditional protocols like NFS, iSCSI or Fibre Channel. The protocol is translated to NVMe at the storage system. While this approach is fine for some environments, others like web-scale hosted services, real-time analytics, business intelligence and high transaction processing may need more.
The storage controller problem leads many organizations with high IO requirements to leverage NVMe flash storage in the servers. The problem is that the internal storage approach creates islands of storage, inefficient capacity utilization, limited data sharing and scalability problems.
Designing a Storage Architecture for NVMe
To exploit NVMe fully, the architecture needs to be entirely NVMe, from the HBA in the attaching server to and through the storage system. However, even with an end-to-end NVMe architecture in place, these architectures need to address the controller bottleneck. The IO load needs to be better distributed.
The E8 Storage Solution
E8 Storage offers an end-to-end NVMe based storage system, which uses NVMe over Fabrics via Ethernet. E8 Storage solves the bottleneck problem by separating the data and control IO paths by installing E8 agent software on each connecting host. E8 Storage claims these agents perform 90% of data operations, significantly reducing storage system CPU responsibilities. As more servers with E8 agents connect to the storage systems, each brings more compute power which enhances the ability to maintain performance, even at scale. Each connecting server can connect to four E8 appliances and each appliance can support 96 host agents.
As a proof point of its performance, E8 Storage recently announced a record-breaking performance benchmark using IBM Spectrum Scale, a clustered file system that provides concurrent access to a single file system, or set of file systems, from multiple nodes. IBM Spectrum Scale is a popular file system in E8 Storage target markets like High Performance Computing (HPC), Artificial Intelligence (AI), Machine Learning, and Analytics. The SPEC SFS benchmark showed a 0.69ms overall response time, 8X lower latency than the previous record and 2.5X more builds than previous IBM Spectrum Scale submissions. The E8 Storage solution delivers more performance on less hardware, consuming only two rack units instead of 20U!
New in 2018
E8 Storage has added new capabilities to make the solution even more appealing. The company has moved from a turnkey hardware solution to a software only model. A vital aspect of this is the recent introduction of newest Intel processors, which allows the software to turn almost any server vendor’s Intel Skylake, NVMe-based systems into an E8 Storage appliance. This capability allows customers to use their preferred server vendors and enables solution providers to build custom solutions for their customers.
The E8 Storage solutions are built on recognized industry standards, including Converged Ethernet with RDMA and standard 2.5” NVMe SSDs. The addition of a software-only model furthers that point.
The primary reason for E8 delivering its solution via branded appliances was the lack standard NVMe offerings when the product was first introduced, especially the dual controller and dual port NVMe SSD support needed for high availability. Now, more dual controller options are being introduced to the market by leading ODMs. E8 Storage will continue to sell its own branded appliances for customers that prefer that approach.
The software update also includes new high availability enhancements within the E8 agent. Customers can leverage their existing disaster recovery schemas with application level replication, or use the E8 Agents with volume mirroring across multiple E8 Storage appliances or third-party NVMe servers with E8 controller software.