It’s Hard Out There for a Pump: The Changing Nature of Content Storage and Delivery

Content distribution and delivery systems, historically referred to as video pumps, have a much more complicated job than ever before. While technology is enabling streamlining of some operations, it is also increasing customer expectations of anytime, anywhere availability. How the creation, distribution and consumption of content is incredibly different than just a few years ago.

The Old Way was One Way

While there is a multitude of ways to distribute and deliver content today, historically this was not the case. Typical distribution companies would create a single cut of a TV episode or movie prior to distributing it. The content had a single maturity rating, such as PG or R. It also came in a single format, such as on either two inch tape reels or ¾ inch video cassettes (pre-film days) for television or DVD, or a number of 35mm film reels for a movie theater.

There was also a single distribution channel. A given piece of content would be sent to a single TV station or a network of theaters. A TV show or movie was distributed once and then the process was over. A movie that finished its first run could be returned to the distributor, that could then redistribute it to a second run theater without any additional content creation work, assuming it wasn’t damaged of course. Occasionally, a piece of content might be seen on an ongoing basis, such as the yearly screening of the Wizard of Oz, Sound of Music or It’s a Wonderful Life. But even then, the job of the distribution company was done once the studio received the movie.

These content delivery needs didn’t create strong storage needs. If a given piece of content is only seen once or at best once a year, it does not need to be stored online for immediate access. Combine that with the fact the cost of storing content online historically was astronomical, and you have the results – no online storage of content that was not currently in demand.

Not only was the content not stored online, it might not have even been stored digitally. It was often cheaper and more reliable to store the master on celluloid or analog videotape. Even once television stations began to broadcast digital content, the digital version of the content was created on demand by the television station. They would do an analog to digital conversion of the content just before screening it, and the digital version would be deleted after the screening. Keeping it online or in digital format simply wasn’t financially feasible, or in many cases even a violation of the distributor’s agreements.

Again, the job of the content distribution and delivery organization was complete once it had delivered the single format to the television station or theater that was to screen the content. Any analog-to-digital conversions or cutting of different versions to different lengths or maturity levels was usually done by the studio screening the content.

The New Way is Every Way

The first change in how content is delivered is that since content is now created digitally on the front-end of the process, it allows the content distribution and delivery organization to perform parts of the process that other entities used to do. For example, they might be asked to create multiple versions of the content for audiences with different sensibilities. Different audiences are more tolerant than others of various forms of content. Consider the film Love Actually, whose original version can only be shown on a premium channel in the US, and cannot be shown in any form in some countries. It features a dozen lead characters in various love stories, ten of which can be shown most any time during the day. But there is “the naked couple,” and the film portrays infidelity and other less-than-mainstream liaisons.

Content delivery companies might be asked to create a “clean” version of Love Actually that – among other things – completely removes the naked couple from the movie and edits out much of Bill Nighy’s dialogue. They also might also be asked to generate a version that is cropped to imply nudity but not show it, as most nudity is not allowed at any time on American broadcast networks. Some countries are fine with mature dialogue and nudity, while others allow one but not the other, and others allow neither.

What is considered mature dialogue also varies from country to country, so a word that might be bleeped out in one country might be perfectly fine in another. The result is that one can easily imagine dozens of different versions of the same movie to fit all different levels of sensibilities around the globe.

Then there is the format question. Where historically there was one format, there are now dozens of such formats. There are different levels of video quality, such as SD, HD, and Ultra HD. (and in the new world of ABR (adaptive bit rate) codecs, ‘profiles’ are set for every piece of content with encoding rates for all expected device and network connections.) (Every end-device has at least one streaming format – MPEG-2 for old set tops, MPEG-DASH for new IP set tops, HLS for Apple, HDS from Adobe and Smooth from Microsoft.) Content distribution channels want their customers to be able to see the content in a format that is best for their platform, without limiting people with lower quality platforms from viewing the content. So the content delivery company needs to create all different levels of video quality.

New Channels Delivering Content

While a given piece of content might historically go to a single distribution channel (e.g. TV or theater), it can now be delivered via a variety of channels. There are hundreds of cable television stations, with different levels of sensibilities as to what they consider to be appropriate content. But nothing has had the impact on content distribution and delivery more than the Internet. Over the top (OTT) video, as it is called, now drives a significant portion of content. Although much content is delivered through only one OTT provider at a time due to contractual negotiations, a single contract negotiation can change which OTT provider is given access, which may require changes in video quality or streaming format. Add to this list of distribution channels the concept of multichannel “skinny bundles”, such as Sling TV, PlayStation, and DirecTV Now. The number of different channels through which a given piece of content can be simultaneously broadcast is mind numbing. Just for fun, go through cable TV and see how many of the exact same infomercials are showing on a variety of different cable channels in the wee hours of the morning.

Where a given piece of content was viewed once and then discarded, today’s content is delivered live (i.e. broadcast or screened) as well as on-demand. An HBO show, for example, is broadcast live on HBO but is also immediately available on HBOgo.com and HBOnow.com. That same piece of content might also be available via Netflix, Hulu, or Amazon and must be available on demand at any time. This means that unlike the old days, all content must be online all the time.

The various streaming channels also use a variety of streaming formats, such as RTSP, HLS, MSFT Smooth, and MPEG-DASH. The more formats that you can support, the more distribution channels you can appeal to. This means that content delivery companies must support a variety of formats. It’s hard out there for a pump.

The New Way Creates New Storage Challenges

All copies in all formats must be available at all times. There may be copies for different distribution channels, including digital formats for content delivery, physical formats such as film or home video, and a near-line or off-line archive of all content for data protection purposes.

With delivery, however, the number of copies goes up even more. There will be the gold copy and the processing pool used to create other copies. This requires both origin storage and a delivery cache that both support multiple streaming formats.

For legal reasons, every customer of a content distribution company must have their own unique copy. This is because each film release network, television or cable station, or OTT provider negotiates how long they are allowed to display a given piece of content. The studio producing the content relies on the content distribution company to enforce this contract by giving the distribution channel its own copy and deleting that copy when the contract is concluded. While it might be possible to do this with a single copy, studios feel more comfortable knowing a given channel’s copy simply doesn’t exist once their contract is over.

These unique needs of distribution and delivery create unique storage challenges that usually results in multiple, purpose-built systems. Doing this is expensive, and creates redundant equipment where redundancy only adds cost.

New Storage for a New World

What if you could have a single storage system that could meet all of the various demands of both the distribution side as well as the delivery side? It would, of course, need to support multiple protocols for the various needs of the many systems that might put data into or out of the system, including FTP, NFS, SMB, and the increasingly popular S3. It would also be great if you could give each channel its own unique address to a single shared piece of content, which could be revoked without actually deleting the original content. (Whether or not studios will accept that as valid is a different question.) You would also need to support multiple different types of media to reflect that, while all data must remain online at all times, all copies are not created equal. Some copies need to be on SSD, where other copies can be on SAS media or even SATA. The gold copy and origin storage probably need to be on high performance storage, as does any data that is currently being processed. Depending on the needs of a particular project, this could be SAS or flash. But the delivery cache is hit the hardest with multiple simultaneous requests for the same media, so it should definitely be on flash.

A storage system designed for the modern era would also need to recognize that companies are now located all over the country and world. It should support a global storage pool that would automatically replicate to multiple locations for both performance and redundancy. It would also need to support different levels of cost and scale, depending on the needs of a particular location or project. If a given process needs block or file access for performance or compatibility reasons, the storage system should be able to support that as well.

StorageSwiss Take

If all of the different versions of the content in all of the different stages of its lifecycle could be served by a single storage system, companies could achieve significant savings by using such a system. Having a single global storage pool allows you to increase your level of protection without significantly increasing your cost. You can have more copies and more location, and use higher levels of data protection that support multiple simultaneous failures. Content will be safer than ever before, more available than ever before, and yet cost less to store.

Sponsored by Concurrent

W. Curtis Preston (aka Mr. Backup) is an expert in backup & recovery systems; a space he has been working in since 1993. He has written three books on the subject, Backup & Recovery, Using SANs and NAS, and Unix Backup & Recovery. Mr. Preston is a writer and has spoken at hundreds of seminars and conferences around the world. Preston’s mission is to arm today’s IT managers with truly unbiased information about today’s storage industry and its products.

Tagged with: , , , , , , , , , , , , , , ,
Posted in Article

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 21,941 other followers

Blog Stats
  • 1,292,328 views
%d bloggers like this: