IT Can’t Ignore the Video Surveillance Data Explosion

IT professionals are simultaneously being pulled in multiple directions. For most, data center management is more like triage than a well-engineered series of processes. As a result, IT managers and CIOs are very careful about which projects they “own”, which ones they advise on, and which ones they ignore. One project that is sure to cross the IT desk is storing and maintaining video surveillance data. While it may not be on the top of their list, IT could provide tremendous value to the organization if they were to own this particular project.

The Problems

Technology is revolutionizing video surveillance, overwhelming the existing infrastructure and expanding its use far beyond the original security purpose. High-resolution cameras are just one example of this technology advancement. Thanks to wireless connectivity and low cost, they can be easily deployed in large numbers. The problem is that the video they capture can be transferred very quickly thanks to high-speed WiFi and the size of the file can be enormous thanks to high definition video. Video retention times can vary widely, from a few weeks to multiple years, based on the organization’s needs and applicable regulations. Then, after all that video is stored, finding a specific few minutes or seconds of video can take hours or days without the right process and systems.

High Resolution Video

The first problem, inexpensive high-resolution video capture, is common across industries. Most readily available cameras today record video at extremely high frame rates that allow for incredibly detailed playback, even if that level of detail isn’t specifically needed. While the resolution can be reduced, there are increasing demands to use the maximum resolution so that applications like facial recognition software can deliver better results. In addition, thanks to wireless transfer and low cost, these cameras can be deployed liberally, multiplying the size of the resulting data sets and creating a major storage capacity issue.

Multiple Cameras

The number of cameras also compounds the second problem, high bandwidth transfer. To keep up, the receiving server and storage system have to sustain very high data ingestion rates. Video does not trickle in, one camera at a time, but streams in continually for all cameras. For many companies this can range from dozens, to sometimes hundreds of cameras and citywide programs easily reach the many thousands of cameras. If not architected correctly, this can lead to multiple servers being deployed to run the required Video Management Systems (VMS), each with their own locally attached storage. Keeping costs in line means using a system that is properly designed and integrated with shared storage.

Data Retention

The third problem, increasing retention times, leads to a storage capacity issue that only IT has had experience dealing with. In the past, surveillance data only needed to be kept for a few weeks. Nowadays, many companies’ or government agencies’ regulations require this data be kept for a year or more, and that’s only expected to increase.

Additionally, organizations are at the early stages of leveraging video surveillance data for more than just security or the monitoring of employees or customers. As with other big data initiatives, video surveillance is now seen as a tool to be used for analytics by marketing. For example, companies are using facial recognition software to identify customers and track their shopping behavior as they visit store locations. It is reasonable to expect that requests for surveillance data will continue to increase.

The retention problem leads to an ongoing concern for any surveillance project, finding budget to store all this information, both initially and for a long period of time. As indicated above the typical answer when IT is not involved is to buy more servers each loaded with disk. In addition to cost, this approach also makes the fourth problem more troublesome, finding data when needed.

Accessing Stored Video

The fourth problem, locating and accessing specific video ‘footage’, stems from poor infrastructure design and an overwhelming amount of data. The disparate storage silos represented by multiple VMSs are difficult to search and the volume of data generated by large numbers of high-res cameras are beyond the ability of these low-performance systems to find that ‘needle in the haystack’. To overcome this challenge, some organizations do set up a central repository with a sophisticated video management software solution. However, this second layer adds to the cost of the storage infrastructure and its complexity. It should only be added if the first layer of storage could be eliminated.

How IT Can Help

IT professionals have dealt with these types of problems before and are well equipped to respond to the video surveillance challenge. Addressing it requires finding the right vendors and implementing the right processes.

The first step is to address the need for rapid capture of data from multiple points of entry, simultaneously, while curtailing costs. This can be done by implementing a high-performance video management solution with a two-tier storage architecture that can move data from high-speed hard disk drives to high capacity tape technology. These two tiers should eliminate the need for local storage on the video capture servers.

Creating a two-tier architecture can dramatically reduce the capacity cost of the video management system and make the increasing demands for retention of this data a non-issue since the cost to store on tape is so low. The ability to move this data between these two tiers should be built into the VMS software.

This software should also offer the capability to provide live viewing via a video wall, as well as sophisticated search and playback. These capabilities will leverage content location information to automatically recall data from disk or the appropriate tape media without operator intervention. Some VMS vendors are now adding intelligent analytics into these systems to further improve their ability to locate and access specific segments of video.

As retention times increase, it is also important that the VMS assures data durability as well as cost containment. The stored data only has value if it actually can be read when needed. This is an area where tape libraries can help, as some have the ability to automatically scan tape media on a periodic basis to confirm media quality. The VMS application could automatically trigger this data verification process on a periodic basis.

Conclusion

IT certainly has enough projects on its plate so adding another one may not be a popular choice. However, there are few projects where IT can have such a dramatic impact on the organization as video surveillance, typically without requiring special skillsets. Video data, while it can consume an inordinate amount of storage, is for the most part like any other data set that needs to be managed. IT can apply its core competencies to the video data management problem and provide the organization with reduced costs and better data value.

Sponsored by Spectra Logic

Spectra Logic, deep storage manufacturer, provides secure, network-attached storage for any IP video surveillance environment, delivering the data integrity, high reliability, simplicity and scalability that these environments demand. Spectra’s Video Surveillance Archive Solutions enable users to safely and efficiently retain years of IP video with low-cost tiered storage. Using Spectra’s open API, any VMS package can seamlessly integrate to Spectra’s BlackPearl interface, allowing automatic video archive on both disk and digital tape and enabling cameras to be utilized to their full potential.

Click Here To Sign Up For Our Newsletter

About George Crump

George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.