A lot of the advancements happening in storage seem to be for unstructured data. Why is that? Have all of the problems for databases been solved? It’s not like databases are no longer important. But what does seem to be true is that unstructured data is definitely more important than it used to be.
There is More Growth in Unstructured Data
First, the growth rate of unstructured data is much higher than the growth rate of traditional database data. Part of the reason is that traditional database data is technically straight text; it’s a series of numbers and letters that are much smaller than the we find in the unstructured world. Also, because databases are “structured” much of the redundancy between the text being stored is eliminated.
A lot of unstructured data is also simple text, but that data is not being generated by machines, sensors and Internet of Things (IoT) devices. These devices create massive amounts of very small data files.
There is other unstructured data, rich media that is seeing growth in both number of files and in the size of those files. In just a few years we’ve gone from SD video to HD video and from that to 4K and beyond. The incredibly low cost of HD and 4K cameras has led to a significant increase in the use of video. The same is true of image data; today’s cameras offer much higher resolution than cameras of just a few years ago. Many of us have more photos on our phones then we would’ve taken in a lifetime.
In addition to machine-generated data from sensors and IoT device, as well as video and image data, there have been an incredible increase in the growth of genomics data. What used to cost tens of millions of dollars and take several years now costs under $1000 and can be done in a day. These cost and time reductions have resulted in an astronomical increase in the amount of this data being stored on modern storage.
There are More Uses For Unstructured Data
Unstructured data used to be the creation of one application, used just for its intended purpose and then stored, never to be accessed again. Consider, for example, security cameras. Their original purpose is to capture data (images of people) and store that data in case something bad happens. But what if this data could have more purposes?
First, with the advances in facial recognition software, the data the cameras capture could proactively warn an organization a nefarious character has parked in their parking lot and that security should be more alert. Security becomes a proactive function instead of a reactive one.
Second, that data can be leveraged for more than security purposes. It can be used by marketing. Again using a facial recognition application that can track individual shopper behavior in a large department store instead of looking for criminals, the store could find out what types of customers go where in the store, how often and how long they look at certain merchandise, and whether they buy anything.
Facial recognition used to be only available in science fiction movies, then to secret government agencies but now it’s available to anyone with an iPhone. It is just one example of how unstructured data is repurposed and reused by the modern data center.
The Problem With Unstructured Data Importance
The problem with unstructured data becoming more important in the data centers is that today’s storage status quo, the legacy network attached storage (NAS) system, wasn’t really designed for the pace at which unstructured data is growing and how this data is now being used. To learn how to break the storage status quo so you can do more with your unstructured data, join my colleague George Crump and Panzura on our on demand webinar, “Overcoming the Top 3 Challenges of the Storage Status Quo”, available now.