|
 |
|
General Information
|
Add To My Personal Library |
July 18, 2008
• Vol.30 Issue 29
Page(s) 23 in print issue
|
Disaster Recovery Testing In Your SME
What Needs To Be Done & When
|
A nearby river floods its banks, and within a few hours, the enterprise’s campus is submerged in knee-deep water. Everyone safely leaves the premises, but the technology-driven company’s employees scatter with few instructions about where to go and what to do. Most of the accounting and billing department’s employees are nowhere to be found, while the call center staff has been told to go home, where they are unable to work. The head of the sales and marketing department instructs her staff to work from home, but most are unequipped to do so. And worst of all, over 90% of the firm’s employees have no idea where and how to access the data and applications to get their jobs done.
Meanwhile, the cost of the downtime is rising exponentially, measured in thousands of dollars per minute. If the firm is not up and running in two weeks, it will likely never recover. But this is not what was supposed to happen; just one month ago, the company’s head of operations signed off on the CTO’s disaster recovery plan, which offered a detailed outline showing how the firm could begin to function again if the enterprise’s infrastructure were destroyed.
One of the key elements was a replica data center that existed offsite, with all of the applications up and running in a matter of minutes following a mass outage. Access is available to anyone online with the login and password information. The problem: Nobody, besides a few people, knows about the disaster recovery plan, much less how to log on to the replicate network.
The scenario unfortunately paints a picture of what would likely happen in many enterprises in the case of a real disaster. Enterprises that are grossly negligent and have no disaster recovery plan in place are increasingly rare. But for those with a disaster recovery plan in place, it’s a safe bet that most fail to regularly test the plan as much as they should to make sure it would work in time of disaster. The issue is simple: Even the most stellar plan is not worth much if it is not regularly tested.
“You don’t want to learn the ropes when it’s an emergency,” says John Matzek, co-chief executive officer of Logic IT Consulting (www.logicitc.com). “It is only after testing that you can then properly set expectations for how long it will take to get services running.”
 Downtime Metrics
A key metric involved in recovering from a disaster is the RTO (recovery time objective), which gauges the amount of time it takes for a data center or department to be up and running in the case of a cataclysmic event. The RTO will vary from department to department, depending on its importance, and the metric should serve as a key benchmark during the testing process, says Adam Waxman, vice president of client services at ROBObak (www.robobak.com). “A call center, for example, should have priority over accounting, and the RTO for the call center might be two to four hours,” Waxman says.
While using the RTO as a standard of measure, the enterprise should conduct tests frequently and view them as a learning process each time, Waxman says. He recommends testing once per quarter. After each test, a review should look at how the SME can improve the process. “The testing process is about making mistakes and learning from those mistakes,” Waxman says. “A lot of people get scared about tests. But you really need to find where the gaps are because that is why you practice.”
Waxman, who has worked with many businesses that have developed data recovery plans, says many SMEs worry too much about getting an alternative site or replica data center up and running and then end their plans there. “A lot of people are not planning for the day when they go back to the original data center,” Waxman says. “But you are supposed to eventually transition back to your original environment or to completely rebuild the original environment, and you must test through that.”
A Group Effort
Disaster recovery testing is one area that absolutely must involve departments outside of IT, yet enterprises all too often do not heed its importance. A good example of how businesses are generally apathetic about testing is fire drills. “We learned about fire drills in elementary school, but now we have forgotten about it,” Waxman says. “Now when the alarm goes off, we close our doors and say, ‘I don’t have to go downstairs.’ That’s really what goes on.”
It is thus necessary to educate upper management about the importance of testing and what they stand to lose if they do not heed your warning. Then, tests should ensure that all users have connectivity for applications that are crucial to the different business areas, says Michael Croy, director of business continuity for Forsythe Technology (www.forsythe.com).
“To verify connectivity, one should have a business person sitting there at that remote keyboard to make sure they can hit that ENTER key and that they are satisfied with the connectivity and the accessibility to the data that they are looking for,” Croy says. “They can also utilize that testing time so that if remote access is different for them than local access, then they know how to do that and use remote access.”
Risk Assessment
Testing needs to be comprehensive and particularly adapted to where the data center and enterprise are located. On a macro scale, the SME needs to consider environmental factors. A location near a river, for example, should involve disaster recovery tests in the event of a flood, while a firm in California might be more prone to earthquake risks and will thus require different testing scenarios.
“One way to do this is to look at the different crises that can befall a company,” Croy notes. “You can have a security lapse, where you have a breach and a virus brings your system to a halt . . . while on the other hand, businesses in the flood plains of Iowa and Missouri have [recently] lost their data centers. It could also be power failures or whatever.” 
by Bruce Gain
Updating Your Disaster Recovery Plan
How often should a small to midsized enterprise update its disaster recovery plan? The answer is simple: every time it does a disaster recovery test. Each test should be a learning process indicating where the plan falls short, so the updated plan thus needs to rectify what went wrong during the test. A general rule of thumb is that enterprises should test their disaster recovery plans quarterly, so the backup plan should be updated accordingly, says Adam Waxman, vice president of client services at ROBObak (www.robobak.com).
“Businesses and their needs constantly change, so companies need to update their disaster recovery plans to reflect those changes,” Waxman says. |
Bare-Bones Testing
Disaster recovery does not necessarily only involve getting data centers up and running after they have been subjected to earthquakes, floods, and hurricanes. Bad things can happen on a much more basic level, as well, such as when a power supply goes down and the server rack must run on the alternative feed. An important disaster recovery test, which SMEs often overlook, involves the power and cabling infrastructure backup systems.
“Most [data center managers] don’t realize that they have not set up, cabled, and installed everything properly until there is a problem and the whole cabinet goes down,” says Calvin Nicholson, director of product marketing for Server Technology (800/835-1515; www.servertech.com).
An important test involves the alternative A and B power in-feeds to a cabinet. Both feeds must be able to individually power the cabinet so that if one of the feeds goes down, then the other can support the full load of the server, Nicholson says. Each feed should thus be routinely tested to make sure it can handle the other feed’s power load in the event of a power feed failure. |
|
|
