All-Flash storage is taking over many, but not all, production storage responsibilities, but because of cost, most organizations can’t justify placing all production workloads on all-flash storage. These organizations then certainly can’t justify placing secondary storage and data protection workloads on all-flash. These systems, in most cases, have the performance capabilities to support all of these various workloads but the high cost of flash relative to high capacity hard disk drives makes the goal of an all-flash data center impractical.
If the cost variable of all-flash systems can be resolved without sacrificing performance then most data centers would happily convert to all-flash. The system would also need to scale from a capacity standpoint to meet the demands of storing all of the organization’s data. Since most scale-out systems, scale compute and capacity in lock-step, it is likely that most organizations would have more available storage performance than they would need.
Most importantly, a single all-flash system that stores all data simplifies storage management significantly. Today, IT administrators often have five to six primary storage systems, two or three secondary storage systems and two or three backup storage systems. They not only have to manually decide what data should be on which storage system but they also have to learn all the various management interfaces for each storage system.
The elephant in the room of course is how can the cost of all-flash storage be driven down to the point that it is competitive with hard disk drives and maybe eventually tape technology? The answer is multi-faceted including rethinking media protection (erasure coding) and data reduction algorithms. It also means effectively using Quad-level Cell (QLC) flash which has a price profile that compares favorably to high capacity hard drives. The problem is that QLC has a relatively low endurance threshold and most data center class storage system vendors have deemed it a consumer only media.
VAST Data promises to solve the QLC problem through a combination of technologies to make the concept of a single all-flash tier a reality for data centers.
Introducing VAST Data
VAST Data launched in 2019 with $80 million of funding. They began selling their solution in late 2018 and, at launch, had several customer purchases of over one million dollars. The goal of their storage solution is to deliver to customers a system that can store all of their data, regardless of classification, on flash storage.
VAST has built a universal storage system that addresses every aspect of the typical data center by delivering all-flash performance and Tier-5 cost efficiencies. The product is designed to deliver exabyte scale and uses NFS and S3 protocols as well as extensions for RDMA.
The VAST solution counts on the reality that most customers don’t need the extreme performance that higher end systems claim and that most customers are already using a file-system for most of their data and anything that doesn’t, can be converted to run on a file system.
The VAST Data solution counts on three new technologies. The first is NVMe over Fabrics, which VAST uses for compute to storage-shelf interconnect, resulting in each node having very low latency access to storage media. The high speed network enables VAST to leverage loosely coupled, stateless compute nodes which connect directly to the storage enclosures. The cache-less, stateless design eliminates the node inter-communication that becomes a scaling problem for typical scale-out storage systems. Each enclosure can provide 18TBs of storage class memory and 675TB (raw) of QLC capacity. Customers can scale performance and capacity independently.
The second technology is storage class memory which VAST uses as a tier to store inbound writes. The storage class memory tier enables VAST to deliver incredibly high performance to the IO type that is most problematic for flash writes. VAST also uses this tier to manage how data is written to the third key technology QLC flash. The key for VAST is the large storage class memory tier gives them the luxury of time to optimize how the system will interact with QLC.
VAST uses a series of methods to maximize the life expectancy of QLC technology. The Storage Class Memory tier, today leveraging 3D XPoint to buffer writes enables VAST to perform full-stripe writes to the QLC tier. This minimizes the write amplification typical in all-flash systems and which is especially problematic for QLC media. The data is also deduplicated prior to being written to QLC, and because the data is stored natively in the storage class memory tier, they can maximize deduplication efficiency. This is something that VAST calls similarity-based deduplication. Since the system is also file system based it can monitor the file types most associated with temporary files and ensure that it does not intermingle that data with more long-term data.
VAST’s data protection capabilities are also impressive. First, it leverages an enhanced data protection algorithm that takes advantage of the solution’s wide striping to provide maximum protection with minimal capacity overhead. Typical raid protection groups are 150+4 which VAST claims is 3 times faster than HDD erasure coded rebuilds with only 2.7% capacity overhead. The 500+8 grouping is 2 times faster than HDD erasure coded rebuilds with only a 2% capacity overhead.
In addition to VAST’s media protection capability the system also provides continuous snapshots capabilities. The system uses a redirect on write technique to provide high performance, with nearly unlimited snapshots. Snapshots are also granular to the file level because of its file system foundation. Snapshots are writable and cloneable and volumes can be rolled back to any protection interval.
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 