“IT we have a problem, we are out of flash.” On several occasions flash NAND manufacturers are telling Storage Switzerland they are nearly sold-out of flash for the rest of the year. A lack of supply means flash storage prices are going to go up for at least the next two years.
Organizations must develop strategies that will enable them to live on less flash for the time being. The good news is that by developing a new flash management strategy, you’ll drive down overall storage costs, simplify your data protection process and improve your data preservation capabilities.
What’s Causing the Flash Shortage?
The flash shortage is being caused by a perfect storm of conditions. First, flash is becoming increasing popular and is now used in everything from smartphones to data centers. Second, the industry is in the process of switching its fabs to some form of 3D NAND technology. While in the long run this will increase capacity and lower costs, the conversion process lowers the amount of NAND vendors can produce. Don’t expect vendors to start building new fabs to meet demand either because they typically cost more than $10 billion to build. Even if every NAND vendor started building one new fab tomorrow it wouldn’t help. It takes about two years to build a fab from initial construction to manufacturing the product. And during that two years demand will rise and the shortage will continue.
What’s an IT Pro to Do?
For enterprises, this shortage should be very manageable, you don’t need as much flash as vendors tell you and your data center does not have to be “all-flash”. Active data typically consumes 10 percent or less of the available capacity in the environment. That means that 90 percent of an organization’s data should be on something other than flash, shortage or not.
The reason the “move to all flash” strategy is so popular is it, in theory, eliminated the need for data management. All data was on the fastest tier possible and that tier delivered more performance than most applications could ever need. All-Flash was an expensive way to avoid the headaches associated with data management but flash was getting cheaper, until the shortage. Now flash is becoming more expensive and harder to get.
Data Management in a Flash Era
Data management is making sure an organization’s data is on the most cost correct storage type based on the use of that data. The concept of data management is as old as the concept of a data center. What made data management difficult was the access latency of the less expensive storage medium as compared to the primary storage medium. If the data being requested is not on the primary storage the time difference to retrieve that data from the secondary storage tier is the latency. Flash’s near zero latency, of course exacerbates this difference. Another challenge is gaining user acceptance which will only happen if there is no change in their workflow.
The Data Management Good News
The good news is the new design of data management architectures can meet both the latency challenge and users expectations of seamlessness. First, flash is still affordable enough that they can over-provision flash capacity. If 10% of data is active, then creating a flash tier or cache that is 20 percent of the total capacity should virtually eliminate any tier/cache misses while still reducing flash expenditures by 80 percent compared to the all-flash data center.
Second, the secondary tier is much more responsive, more cost effective and more scalable. For environments where second tier latency is critical, a private cloud / object storage system that is on-premises should more than address any latency concerns. For environments where latency is less of a concern, the public cloud offers a viable, cost effective option that also reduces data center floor space.
Finally, solutions exist to address the seamless transfer of data between high performance flash storage and private/public cloud by either integrating all the media into a single platform or connecting multiple platforms together.
The flash shortage is very manageable, and the skills that organizations develop to survive it will lower long term costs while improving data preservation. Storage Switzerland recently completed an on-demand webinar “Use Cloud Storage to Overcome The 3 Challenges to ACTIVE Data Archiving“. Click here to watch now and learn how to survive the flash shortage.