Big Data for Small Enterprises

Big Data is not just for big enterprises. Small to medium-sized enterprises (SME) can reap the same rewards. But SMEs can’t build a Big Data infrastructure like larger enterprises. The good news it they don’t have to, they can leverage the compute capabilities of the cloud. The challenge is how to get the right data to the right cloud at the right time.

Big Data Value

Big Data can come in many different forms, such as Internet of Things (IoT) devices, log data from internal systems or a correlation of data from multiple legacy applications. Organizations can use this data to help with decision making, problem solving or new product design.

Big Data has two specific requirements that have in the past excluded SMEs from participation. First, in most cases, Big Data requires a scalable storage infrastructure to accommodate all of this source material. Second, it requires a scalable compute architecture so that data sets can be processed quickly enough to make near real-time decisions.

The Cloud Conundrum

The default answer for a scalable compute and storage infrastructure is the public cloud, which can provide an almost limitless quantity of both. But the cloud may not be the most ideal location to store data. Data has gravity and while the ability to scale-up is required, it is very rare to “scale-down” storage. The result is the periodic cost of storing all of an organization’s data in the cloud over a long period of time becomes prohibitive. Additionally, on-premises object storage offers very similar ease of management and expansion while being less expensive over time.

The compute side of Big Data processing does scale-up and scale-down. Typically there is a set moment where a result or answer is needed, a range of data needs to be processed as quickly as possible so the organization can make a decision. An ideal public cloud use case, if the data is there.

The advantages of cloud compute forces many organizations to also use cloud storage so processing can occur quickly. As a result SMEs are forced to go “all-in” on the cloud even though they’d prefer not to.

The SME Cloud

For larger application datasets, the ideal SME cloud is compute only, with data loaded on an as needed basis, essentially caching data from on-premises storage to the public cloud compute. The design is the exact opposite of the typical cloud gateway where most of the data is in the cloud and active data is cached on-premises. The SME Big Data Cloud keeps data on-premises and then caches data temporarily to the cloud for processing.

The SME Big Data cloud is also unique in that the cache has to be more sophisticated than the typical first-in, last-out technology. While that approach is fine for a default action, SMEs need the ability to override the behavior and initiate compute on-demand by caching data needed for processing next to cloud compute. For example, a small, independent research organization could leverage unlimited compute resources to support grant cycles and complete necessary work without infrastructure investment.

StorageSwiss Take

When used correctly the cloud can be the great equalizer, enabling the SME to go toe-to-toe with larger enterprises. It is key, though, that the SME leverage just the aspects of the cloud that they actually need to optimize costs and derive the most value.

To learn more about how SMEs can better leverage the cloud for their Big Data needs watch our on demand webinar, “10-Minutes to Cloud: How to Quickly Shift Big Data Processing“.

Watch On Demand

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , , , , , ,
Posted in Blog

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 21,941 other followers

Blog Stats
  • 1,316,117 views
%d bloggers like this: