What’s the Next Big Thing in Storage After Flash?

Flash storage is fundamentally changing the data center. It is allowing databases to meet the demands of users and virtual infrastructures to achieve new levels of virtual machine (VM) density. But what comes next after flash? Is it another advancement in memory storage, or is it something else?

Certainly there is another wave of memory technologies to follow. These will provide DRAM type of durability with flash persistence. But these memory technologies, while faster, will not provide the step level jump in performance that flash brought over hard disk drive storage.

Instead, the next big thing in storage is going to be mobility.

Here is what we know. Flash is definitely going to become the dominate storage tier for all primary data. As prices continue to fall, it will also become the primary storage medium for all secondary data.

However, flash will not be able to replace high capacity disk or even tape for long term cold data storage. We also know that the primary storage tier will be made up of different types of memory based storage not just flash. In addition, primary storage will be in a different location in the data center, not all of it will be shared on a storage area network. Finally, we also know that the data center will need to use all of these different storage types and locations in order to meet an increasingly demanding set of user expectations and a continually flat IT budget.

Data Mobility – The Next Big Thing in Storage

We have spent years in storage fighting the wrong battle: trying to consolidate all of our data onto a single storage system that has to meet all of our needs. The problem is that it never does. The organization ends up with a system that is a jack of all trades but master of none. Software defined storage (SDS) was a solution to solve this problem, but it didn’t. SDS consolidated the management of storage, but still imposed data walls between actual systems.

Data mobility breaks down walls and allows data to flow between systems based on policies set by the organization. Data mobility also allows the organization to buy the best storage system available and have the data automatically flow to it as it makes sense. The result is that the organization manages data instead of managing systems.

With data mobility in place, IT can add new storage systems and then not have to do anything. As soon as a new system comes online the data mobility solution can analyze it, determine its capabilities and then place the appropriate set of data on it.

Conclusion

Data mobility has the potential to fundamentally change the data center, even more so than flash did. Flash’s impact was only on high performance storage, where data mobility promises to impact all tiers of storage an all types of storage. In a future entry, we will discuss data mobility’s impact on scale-out storage. After all, once you have data mobility in place, do you really need scale out storage? Why not just use multiple scale up systems and let data flow between them?

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , , ,
Posted in Blog
4 comments on “What’s the Next Big Thing in Storage After Flash?
  1. Tim Wessels says:

    Well, Mr. Crump is right in stating that all primary or transactional data will be on flash, and as the price for flash continues to decrease, all of the use cases for secondary object storage will eventually be on flash. Flash becomes the default choice for storage when there is about a 5x difference in the per GB price for flash vs the per GB price for spinning disks. This could happen in the next 12 to 18 months.

    Mr. Crump is less right if he thinks object storage on capacity HDDs has much of a future beyond say 2025. HDDs may go away sooner than that, which means that the manufacturing of HDDs could stop by 2025.. The largest capacity HDD you can buy today is 10TB. Flash SSDs are already shipping at 15TB. Capacity SMR HDDs are illogical for use in anything but an object store, HAMR HDDs are not shipping, and their durability in production storage environments is not proven in the real world.

    As for tape, it looks like tape will be with us until the end of time. You can use LTFS formatted tape cartridges in object storage environments. Spectra Logic does with their DS3 Black Pearl appliance and a robotic tape library.

    Data is not mobile. Devices are mobile. Data is sticky and doesn’t get moved very often, but you can migrate data from a more expensive tier to a less expensive tier using automated data management. Komprise is doing some interesting work on this with their Komprise Observer appliance plus their SaaS management platform and object-based storage anywhere you happen to have it.

  2. Mark Cree says:

    This article describes exactly what infinite.io does with our Network-based Storage Controllers. Our Controllers allow active and inactive data to automatically flow between primary storage systems and an object store or low-cost NAS tier based on IT defined policies.

    Since we are a networking device, we don’t need to continually scan storage systems for changes in file status and therefore don’t leave behind symbolic links or sub-files that can create data management nightmares.

    The infinite.io approach is totally transparent and also doesn’t introduce a new file system, mount point or gateway.

    So yes, data will be mobile.

  3. fstevenchalmers says:

    “…These will provide DRAM type of durability with flash persistence. But these memory technologies, while faster, will not provide the step level jump in performance that flash brought over hard disk drive storage.”

    I respectfully disagree. As soon as an application can store to persistent memory locally, and store to persistent memory in a separate, independently failing location as well, it has everything a disk array needs for durability (just not the disks). There is no need for a traditional software stack, or the latency or host CPU consumption of that stack. That’s the next order of magnitude latency and IO/sec improvement past NVMe over Fabric.

    That having been said, there are small matters of software development and backward compatibility which suggest that this is simply a new segment, and the storage business as we know it continues evolving slowly, as always.

    @FStevenChalmers
    (speaking for self, works for Hewlett Packard Enterprise)

  4. Douglas says:

    Indeed, new storage media will continue to emerge, and making the best use of storage resources will require data to be tiered across different types of storage. 
    Tim is right, you don’t want data to move unless it has to. But the value of data changes over time and we believe data mobility across any type of storage will enable Enterprises to adopt a wide range of storage types (including flash, SAN, NAS, object and cloud storage), to meet their changing business needs.
    With DataSphere we put the data in the right place at the right time to meet the business needs of the applications.

Comments are closed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,047 other followers

Blog Stats
  • 1,444,775 views
%d bloggers like this: