Compression has been available in one form or another for decades, but most of the time compression is applied as data is moving off of primary storage. For example, most backup software or hardware compresses data as it moves into the backup storage area. Current compression of active production data has had too much of a performance impact associated with it. Permabit recently upgraded their data efficiency software and now includes an advanced compression capability, called HIOPS™ that may bring inline data compression to much broader use cases.
Who is Permabit?
Permabit Technology is a company that most IT professionals won’t interface with directly. Similar to the old BASF commercial, they don’t make the storage, they make the storage better! In the Permabit use case, they make storage more efficient by providing data efficiency software (deduplication, compression, thin provisioning and replication) that vendors can integrate quickly to deliver these capabilities to their customers.
Multi-core Data Efficiency
The ability to split data streams apart and use multiple cores to apply its data efficiency techniques is one of the key Permabit differentiators. Most data efficiency solutions cannot stripe their processes across multiple cores. By striping across cores, Permabit can allow vendors to configure an array that can sustain over 1 million IOPS with data efficiency applied, just by taking full advantage of the multi-core processors these systems already have installed. The ability to efficiently leverage multiple cores allows data efficiency services to run inline and not impact the user/application performance experience.
What Makes HIOPS Different?
Multi-core capability enables Permabit’s first key differentiator for HIOPS; its tight integration with Permabit’s deduplication engine is the second. When new data is written to the storage system, it is checked for uniqueness first (i.e. deduplicated). If the data is not unique, then it does not need to be compressed. If it is unique, then compression is applied.
Permabit is unique in that they are the first vendor to perform the deduplication check before compressing the data. They can do this because their deduplication engine is fast enough to perform a check inline without any performance degradation. By removing the duplicates first, the amount of data that is left to compress is dramatically reduced. Even with the overhead of adding compression (the compression process also leverages the multi-core architecture so it executes quickly and inline), overall system performance may actually increase as the amount of data finally written is dramatically less.
In addition to a more intelligent integration between deduplication and compression, HIOPS is also unique because it does not require a separate garbage collection process to free up blocks on the storage system. Instead, the return of capacity happens immediately.
Finally, multiple compressed blocks are packed into 4K blocks (as many as 14 per block) and then stored. When data is read, the compressed blocks are read into cache and the requested data is retrieved. The remaining compressed data stays in cache for rapid retrievals eliminating the need for another read of the remaining data into cache. This makes HIOPS ideal for online transaction processing (OLTP) database environments, as well as other more traditional environments like virtual servers and desktops.
What Does HIOPS Mean For All-Flash Arrays?
Permabit has demonstrated that their deduplication engine can sustain over 200,000 IOPS per processor core. That means that an all flash array capable of delivering 1 million IOPS can now add deduplication and compression without impacting performance.
The all flash vendor would simply integrate an additional quad-core processor to the flash system or just better leverage the cores they already have. Given the relative affordability of processing power and the premium cost of 1 million IOPS capable storage, hiding a quad-core processor in the cost of the system would be relatively inexpensive compared to the return. By doing so, the vendor could then offer 5X or so the storage capacity at no additional cost without any performance penalty.
Storage Swiss Take
While they don’t sell directly to IT professionals, Permabit is a company that IT professionals ought to pay attention to. For storage managers, watching Permabit is similar to server managers watching Intel. If you want to know what your storage system will do next or what it should be able to do next, keep your eye on Permabit.
HIOPS is an excellent example of this. While most of the market is busy implementing deduplication, Permabit has gone back and improved compression, so that vendors leveraging Permabit’s technology can deliver maximum data efficiency with an engine that performs at flash speeds without performance impact.
Permabit Technology is a client of Storage Switzerland