Amazon S3 and Glacier can be used for a variety of use cases including disaster recovery copies, archive or content distribution. But what is most interesting is using Amazon Web Services like Redshift, to replace data warehouses with a more powerful solution that is an order of magnitude less expensive. The key issue for businesses looking to use the cloud, is getting their data transformed and into the Amazon cloud as consistently as possible.
Attunity’s CloudBeam software as a service (SaaS) platform helps resolve that issue by optimizing the transfer of data over network connections. It removes the Extract, Transform, Load (ETL) roadblock, that prevents companies from taking full advantage of an environment like Redshift.
As we discussed in a recent briefing report, CloudBeam does this by optimizing the data being transferred and by performing change data capture (CDC) so that only the unique segments of data are sent. The result is more efficient transfers between private data center environments and public cloud infrastructure.
At Amazon re:Invent, Attunity announced enhancements to the CloudBeam solution. The solution now includes support for Amazon Relational Database Service (Amazon RDS), and leverages Attunity’s high-performance TurboStream CDC and DX technologies for moving and synchronizing data to, from and between enterprise data centers and cloud environments.
Companies looking to leverage Amazon’s Redshift for data warehousing and analytics will find CloudBeam 2.0’s loading performance has been further optimized to enable faster loading speed. These capabilities were on display when I met with two CloudBeam customers at the Attunity booth.
Adam Haines, a Data Architect at Federated Sample, was looking for a way to move the organization’s 1.7TB SQL farm to Redshift for data warehousing. Adam started with a home-grown solution that he designed but he was the only one that knew the code. While certainly good for job security, Haines knew that his scripts wouldn’t scale, didn’t have error condition handling and being the sole person supporting the solution had its obvious drawbacks.
Haines recently turned to CloudBeam to automate the solution. Now Federated Sample has a solution that the team can use, handles data transmission errors on the fly and most importantly, performs the ETL process much faster than before.
We also spoke with Daniel Heacock, a Senior Business Systems Analyst at Etix. Like Haines, Heacock was looking for a solution that would allow them to ETL an Oracle environment into Redshift. They started with Attunity and within two weeks they had fully migrated to the Amazon environment. They also leverage the CDC ability of Attunity to keep their Redshift environment up to date. The move to Redshift allows them to run reports they simply couldn’t produce in the past and do so at 1/10th the cost.
Storage Swiss Take
The ability to leverage the Amazon Cloud for data warehousing, business intelligence and analytics is a game changer. Reductions in cost versus traditional data warehouse solutions are massive combined with the ability to do things that just couldn’t be done before. The challenge is making the ETL process quick and efficient and for that, it is obvious that CloudBeam deserves strong consideration.