Tegile Briefing Note
Of all the “modern” data center applications, Splunk seems to be one of the more popular among more traditional data centers. This is due to Splunk’s specific use case: log data from machines. Almost every enterprise has them and has had them for years. They don’t need to start an Internet of Things (IoT) project or launch a big data initiative. This is data they already have. Splunk gives organizations the ability to quickly analyze this data allowing it to deliver greater value.
The problem is that also creates a unique storage challenge. The data it analyzes tends to be made up of millions if not billions of small files which combined, can require dozens if not hundreds of terabytes worth of data. And Splunk actually makes an admirable attempt at managing this data. It typically will move data between one of three buckets; a high performance but expensive memory based bucket, a less expensive but moderately performing bucket and then a very inexpensive, high capacity but low performance bucket.
While admirable, managing data may not be the best use of your Splunk resources. You’d rather have those resources focused on analyzing more data, more rapidly instead of managing and moving data. Also the time it takes to move this data from one bucket to another adds latency because the entire data set has to move from one storage system over a network to another storage system.
Tegile has an “app” for Splunk, allowing Splunk to offload data management tasks to Tegile’s storage management software. This allows Splunk to focus all of its compute resources on data analysis. As far as the Splunk application is concerned, all its data is on a flash tier all the time. Tegile handles the movement of data between flash and hard disk based storage tiers. It is important to note that the data movement is internal to the storage system itself, meaning that to move data from a hard disk tier to a flash tier does not require the data to traverse a network. To the user this means a much faster acceleration of an older data set.
Splunk also benefits from Tegile’s metadata acceleration technology. Splunk, like any high file/object count environment, is laden with metadata I/O. Tegile’s acceleration of that I/O leads to significant overall performance gains.
Tegile’s app for Splunk allows a Splunk administrator to manage Splunk data directly from within the Splunk environment. It even identifies which Splunk projects are on which Tegile volumes.
StorageSwiss Take
While many of the modern “apps” started with the concept of either using local storage or performing their own storage management, we are continually finding that these apps actually benefit from dedicated, shared storage technology. Tegile’s technology is directly beneficial to Splunk projects, offloading the data management responsibilities and providing metadata acceleration. Tegile’s Splunk app provides Splunk admins with an in-app method of managing projects.
George Crump is the Chief Marketing Officer at VergeIO, the leader in Ultraconverged Infrastructure. Prior to VergeIO he was Chief Product Strategist at StorONE. Before assuming roles with innovative technology vendors, George spent almost 14 years as the founder and lead analyst at Storage Switzerland. In his spare time, he continues to write blogs on Storage Switzerland to educate IT professionals on all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought-after public speaker. With over 30 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS, SAN, Virtualization, Cloud, and Enterprise Flash. Before founding Storage Switzerland, he was CTO at one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection.
StorageSwiss.com - The Home of Storage Switzerland 