One of the biggest challenges facing an organization contemplating a move to the cloud is ingestion. How do you get all your data to the cloud in a timely manner? Sure you could replicate it, but if the organization’s data is of any size at all that process could take a very long time. To solve that problem Amazon has a service it calls Snowball, which is essentially a ruggedized storage container that it can ship to your site. While effective it simply is not enough for some of Amazon’s larger clients. At re:Invent Amazon brought out a truck to solve the problem.
For really big data relocation Amazon is announcing Snowmobile at its re:Invent user conference. AWS Snowmobile is a ruggedized, tamper-resistant shipping container 45 feet long, 9.6 feet high and 8 feet wide. Once connected to your data center it appears as a 100PB NFS mount. It supports 1Tb/second of data transfer spread across multiple 45Gb/second connections. Think of it as 100PBs of storage capacity on a truck that Amazon sends to your data center. Each Snowmobile is waterproof, climate-controlled and consumes about 350KW of AC power (AWS can supply a generator if you need it).
At the conference, Amazon is also announcing an update to the original Snowball: Snowball Edge. Snowball Edge is a 100TB unit ideal for smaller customers looking to migrate data to the cloud. But as bandwidth continues to increase and software gets more sophisticated that use case is subsiding. Where Amazon sees a growing use case is remote facilities. Most organization’s manufacturing facilities are no longer close to the data center but these facilities are powered in large part by robots, all of which generate TBs of data per week. IT can take a Snowball Edge to these locations on a regular basis for data collection, so data can then be analyzed in the cloud.
The initial focus of Snowball Edge and Snowmobile seem to be data import, getting more data to the cloud. But Amazon recognizes the potential to use these devices for rapid export, like in a disaster recovery. An example would be an organization using one of Amazon’s partners for Disaster Recovery as a Service (DRaaS). In the event of a disaster the organizations could bring mission critical systems up in the cloud, then ship the bulk of the data to their new data center by one of the snow options.
The cloud offers a lot of potential, once you get your data to it. The cloud also creates a lot of problems if you need to extract your data from it. There are solutions from various Amazon partners that can do small data shifts through software. And thanks to improving technology as well as increasing bandwidth these solutions are able to handle larger and larger data sets. But, at some point there is simply too much data or situations where moderate amounts of data needs frequent moving. In these cases Snowball Edge or Snowmobile seem to be the most logical solution.