Recovering from a Hurricane

Disaster Recovery Plans (DRPs) should be relatively generic with a design that allows it to survive a data center outage of any type. But each type of disaster requires a unique response. At the Storage Decision’s “Modernizing Your Disaster Recovery Plan” workshop, we discuss each type of disaster and how to prepare for it and recover from them when they occur. One of the most unique is a hurricane. It has the potential to impact a large number of data centers. All our Modern Disaster Recovery workshops cover a variety of disaster situations. But our workshops in Miami, Philadelphia, New York and Houston will pay special attention to hurricanes.

The Good News

From a disaster recovery perspective, hurricanes have one advantage over other types of disasters: because of advances in weather forecasting we can know days in advance that a threat is imminent. Compared to other disasters like tornadoes or earthquakes that give no or only a few hours warning, knowing of a threat days in advance is a nice luxury to have.

The Bad News

The bad news with a hurricane-class disaster is the effects are often devastating and far reaching. Most cities suffering a direct hit from a hurricane of any size will not be data center ready for weeks. This means the organization needs to be prepared to run at the DR site for a long time. The other problem with a hurricane is that they change their mind. Knowing “when” to declare is critical. If the declaration is made too soon and the hurricane veers into a different direction, the IT administrator wastes a lot of time, effort and re-sync time. If the declaration is made too late, systems or data may be lost.

The other challenge a hurricane brings is the state of the original data center. There is a good chance the hurricane will not touch the original data center, either because it veered off course or simple luck of the draw and the organization’s data center is not in the path of the storm. In either case, while the data center may be intact it may not be accessible, even remotely, as power and communication links are often down for days in the wake of a storm. This means IT will have a massive data re-synchronization task on its hands.

Preparing

Preparing for a hurricane is similar to any other disaster. IT needs to make sure that it is protecting data frequently and transferring it to the DR site quickly to meet the Service Level Objectives (SLO) set by the organization.

The workshop also covers SLOs and how to create them. Again, hurricanes provide that advanced notice. This means that you can plan for the switch over to the secondary site. It is possible to perform final and complete syncs on all data protection tasks, so clean copies are available in the secondary location.

The Recovered State

The recovered state is the point of the disaster recovery process where IT operations are fully running at a secondary location. The secondary location could be a site owned by the organization, a managed service provider, a co-location site or a public cloud provider. In the workshop we also discuss how to choose between these various options. The only unique aspect a hurricane brings to the recovered state is the amount of time the organization may host its IT operations from there. The range of time could be very short because of a near-miss or it could be very long because of a direct hit. Both situations have their problems. A short amount of time in the recovered state means a difficult re-synchronization process. A long time means a massive data transfer to new equipment as well as a data center that is able to support IT operations for weeks. Something that the public cloud in particular may not be good at.

Returning to The New Primary

The final step is returning to the new primary data center. The state of that data center will impact how the recovery occurs. Typically a hurricane creates one of two scenarios. In the first, the data center is the victim of a direct hit. From a recovery standpoint this can actually be considered a preferred outcome.

In the face of total destruction, employees and users are much more patient during the recovery process. Also IT does not need to worry about what data remains on site. There is no concern over proper synchronization of data. The downside, of course, is that all the data must be recovered. It is important that IT knows what is most important to the organization.  They need to know which applications to recover first, what the interrelationship between servers and the application are. Finally, they need to understand how much data can feasibly be recovered in a given time frame.

Conclusion

Surviving a disaster is all about planning and practice. While making sure the secondary site has the latest data set is easier, hurricanes present some unique scenarios that IT needs to work through to ensure they can handle. At our Modern Disaster Recovery workshop, we will go through all of these scenarios to make sure you are ready to recover from any type of disaster.

Twelve years ago George Crump founded Storage Switzerland with one simple goal; to educate IT professionals about all aspects of data center storage. He is the primary contributor to Storage Switzerland and is a heavily sought after public speaker. With over 25 years of experience designing storage solutions for data centers across the US, he has seen the birth of such technologies as RAID, NAS and SAN, Virtualization, Cloud and Enterprise Flash. Prior to founding Storage Switzerland he was CTO at one of the nation's largest storage integrators where he was in charge of technology testing, integration and product selection.

Tagged with: , , , , , ,
Posted in Blog

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 22,236 other followers

Blog Stats
  • 1,554,222 views
%d bloggers like this: