Online Analytics Company Leverages Isilon’s Scale Out Storage for Big Data and Virtualization

One of the big challenges faced by today’s data center is dealing with the mixed workloads that applications like analytics, back-office operations and server virtualization present to the storage infrastructure. In many instances IT professionals are faced with having to use multiple storage systems to accommodate these different workloads. This of course drives up costs and makes management of the total infrastructure more difficult.

Storage Switzerland recently sat down with a leading online web analytics company to discuss how they chose Isilon to be the single storage platform for all of their disparate needs. This company, like a growing number of others, has all three data types, back office applications, analytics and server virtualization. As we’ve written in the past, Isilon’s unique scale-out architecture allows the use of a single storage volume across all applications for ease of management and increased efficiency, while at the same time maintaining adequate storage performance.

The production IT department for this online analytics company originally went searching for a storage system that could handle its significant CIFS protocol traffic demands caused by storing the results of data collected from monitoring and analyzing Internet-based social media and websites. Data ingest was a significant problem, as was the inability to scale the system from both the performance and capacity perspective to meet the onslaught of data collection and analysis.

They selected Isilon for its scale-out architecture, which allowed them to seamlessly add capacity and linearly scale performance. The system was quickly put into production and the customer demand was satisfied.

Soon after the Isilon system was brought online an initiative in the research and development group was kicked off to examine the potential impact of using server virtualization within that department. The R&D group’s responsibility was to develop and test the software applications that the production analytics department would eventually use. As a result they needed to spin up and spin down servers at will, as well as push the test code beyond the limits of what the normal production environment was likely to do. This meant that R&D often had a more demanding requirement of both server virtualization and its storage infrastructure.

The R&D department manager knew they had to have a storage system that would not add significant complexity to the server virtualization environment. He wanted to make sure that any capital and efficiency gains made because of this move would not be lost in a more complex storage infrastructure.

Based on experience gained at a prior company the R&D department manager was well aware of the challenges caused by block-based storage in virtualized server environments. The difficulty in having to manage multiple LUNS and allocate them to the appropriate hosts was something he wanted to avoid. As a result NFS was the protocol they chose.

As he explored other NFS options, the R&D manager quickly realized that in many cases you have to subdivide those storage systems into multiple volumes in order to maintain adequate performance, which essentially re-created the block level problem described above. Isilon gave him the advantage of using a single volume on a single storage system that could scale infinitely and keep operational costs from growing at the pace of capacity.

Like any startup project however the primary challenge was keeping costs and administrator involvement low. They could not initially go out and purchase a storage system for the task until the project had proven itself. With this in mind the R&D manager turned his attention toward the existing Isilon storage cluster. The R&D team was able to use the free space on this system and cost-effectively add six more nodes to support the server virtualization project. The Isilon cluster proved itself more than capable of handling a raw bandwidth application like analytics, as well as a random I/O application like server virtualization, without impacting either department’s performance.

Implementation

The details of implementation for this particular company are different than others since they already had Isilon in-house by the time the server virtualization project started. In speaking with the owners of the initial project, that implementation went very smoothly and was quickly receiving over a terabyte of analytics information per day.

Since their Isilon clusters had the aggregated performance required to support mixed workloads from a single volume, implementation for the server virtualization team was very straightforward. They were granted access to the Isilon cluster and then simply started storing their virtual images to it. So, at least initially, there was no implementation time, just leveraging a resource that already existed in the data center.

This is a critical differentiator and advantage for Isilon as it demonstrates the system’s ability to support mixed workloads and, at least in the case of this online analytics provider, to support a completely new project with little to no additional storage costs. It also showed the value of Isilon’s single volume concept. Data did not have to be ‘un-allocated’ from current LUNs and re-partitioned to the new project.

Due to the success of their server virtualization project, as well as the ability of the Isilon storage cluster to provide adequate performance and utilize existing assets, the project quickly progressed from the R&D phase to full-scale production. As that happened, nodes were added to the cluster to support the capacity requirements of the growing virtual machine population. In adding this extra capacity, increased aggregate performance ‘came along for the ride’, scaling in-line with the capacity being added.

One of the real unexpected gains that leveraging the Isilon storage cluster created for this online analytics company was how well thin provisioning worked. VMware, their server virtualization platform of choice, does thin provisioning by default when using NFS mount points. Isilon further helps by providing the performance necessary to handle the dynamic allocation of data blocks as the VMs’ volumes grow. This allows the organization to build VMs at size without having to pre-buy the actual storage capacity, which once again saves them money.

The R&D department’s server virtualization project was so successful that the organization’s IT department began to use server virtualization to meet other requirements, such as back office database applications, email and file sharing, further reducing cost and complexity in their IT environment. Based on the success of the analytics department and the R&D department, the back-office IT department also selected Isilon for their primary shared storage infrastructure.

As each of the three environments grew they decided to have separate clusters. However, each of the clusters can be managed and monitored from a single point, once again reducing operational costs as well as keeping capital costs in-line.

 

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , , , ,
Posted in Case Study

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,157 other followers

Blog Stats
  • 1,496,000 views
%d bloggers like this: