The demands of the modern Media and Entertainment (M&E) organization is pushing storage infrastructure to the breaking point. M&E generates more data, requires faster processing, faster delivery of that data and has, by far, the most justifiable means for retaining data for a very long time. These demands defy all the standard best practices in storage, M&E IT professionals need to use multiple storage technologies, likely from multiple storage vendors to solve their storage challenges. Organizations outside of M&E need to spend some time studying the space to prepare for the future because what M&E IT is dealing with today will be what traditional IT is dealing with tomorrow.
Lesson 1: All-Flash Won’t Cut It
The first problem that M&E IT has had to overcome is how to rapidly process a relatively small amount of data as fast as possible. While M&E is one of the few industries that can actually push Intel CPUs to their limits, their primary bottleneck remains storage. High performance, flash based architectures rule the day here but unlike traditional IT M&E can’t afford to put ALL of their data on flash, they have to manage it and move it to alternative storage at just the right time.
While unstructured data growth is unprecedented, traditional IT is only at the beginning of the problem, thanks to the Internet of Things (IoT), sensors and surveillance cameras, data growth in non-M&E organization will likely grow at greater than 50 percent per year over the next 10 years. Very soon these organizations will not only not be able to cost justify a flash only storage infrastructure, they may not even be able to get it. We may face a situation where the industry can’t produce enough flash to keep everyone happy.
Lesson 2: Archive will Evolve to Preservation
When IT professionals hear the term “archive” they think of the closing scene in Raiders of the Lost Ark, where the Ark of the Covenant is stored in a government warehouse never to be seen again. But data is different now. What M&E learned is there is a monetization and a productivity gain opportunity in keeping data for a much longer time than we ever thought necessary. But this data needs to be accessible and it has to be readable. In other words; it needs to be preserved.
From a storage perspective, data needs to be on a system that can not only cost effectively store data for a long period of time, the system you use to preserve the data also has to be able to deliver that data back quickly when users need it and the system has to make sure that data is in the same state as when it was originally written.
These are just two lessons IT can learn from M&E, there are also the lessons of data distribution and data delivery. You can get more detail about performance and preservation as well as distribution and data delivery by watching our on-demand webinar “Preserve, Distribute and Deliver – M&E’s Three Biggest Data Challenges“.