When most people think of Continuity, they think of bringing a business back online after a fire or a flood. They think of the chemical explosion in West Texas or the businesses destroyed in Hurricane Sandy. They think about the traditional, “smoking crater” scenarios. As such, responsibility for Continuity planning is typically held in a Security or Emergency Management group.
Conversely, when people think about cyber-attacks and cyber Incident Response, if they think about them at all, they typically think about scam emails, virus warnings, or Microsoft patches. If you work in a particularly in-tune or high-tech firm, you might even be aware of a group of people working in a dark room practicing cyber ju-jitsu to keep your network safe. But what “those folks in IT” do and what they need to do their jobs effectively is probably still a mystery.
Before now, the planning efforts for these two activities were rarely coordinated. That is starting to change as more and more business functions go online and more and more companies become reliant on technology to perform their critical functions. Just in the past year, we’ve seen half a country’s power taken offline, major online commercial brokers go down, and two major US government agencies take whole information systems offline.
- In February, Hollywood Presbyterian Hospital in Los Angeles ran without access to email and electronic records for nearly 10 days due to a ransomware attack.
- Around Christmas time last year, tens of thousands of people in and around Kiev lost power for several hours due to a suspected Russian-backed cyber-attack .
- In the fall of 2015, major retailers including Netflix, PayPal, and EBay were all taken offline due to various datacenter issues .
- In July 2015, the Joint Chiefs of Staff unclassified email system was offline for weeks, due to a “serious” cyber-attack .
- In June 2015, the now-infamous hack of the US Office of Personnel Management (OPM) took its Electronic Questionnaires for Investigations Processing (e-QIP) system offline for a number of weeks, effectively stopping the agency from opening any new background investigations .
These kinds of issues can affect cloud-based and third-party hosted data just as acutely as individual businesses and local data. Per a recent study by Emerson Network Power and Ponemon Institute, the second most prevalent cause of unplanned data center outage in 2015 was cybercrime – specifically Distributed Denial of Service attacks. (The #1 cause of outages remained the failure of Universal Power Supply systems, which are a favorite mitigation recommendation of Continuity professionals).
In addition to a data center outage having a negative impact on a business’s image and critical functions, as we discussed last December, these outages can have a real impact on a business’s bottom line. The average cost of a data center outage in 2015 rose to nearly $9,000 PER MINUTE of downtime. The average outage costs $740,357, with maximums in the millions – an 81% increase over 2010 .
So, what can be done?
To steal an idea from fellow Emergency Management professional Tim Reicker, true organizational resilience is really a coordination point between a number of functions that, all too often, operate in a vacuum or silo.
First and foremost, risk assessments and business impact analyses must be kept up to date as businesses and organizations flatten out and put more and more of themselves online. Continuity and availability concerns should feature prominently in your questioning of third-party vendors and in your requirements analysis of new software and services. An inaccurate assessment of your interdependencies or your maximum tolerable downtime can cost you in both tangible and intangible ways.
From there, there’s a lot to keep in mind.
- You must ensure that plans are in place and up to date to both identify and address cyber incidents as well as all other hazards (natural, man-made, and technological).
- Procedures and decision matrices should be written for disconnecting major systems from the internet, going without a key vendor for some period of time, and communicating with your stakeholders. (E.g., what is your organization’s tolerance for data loss? Are there any single points of failure in my supply chain?)
- Have a plan ready for communicating with your customers why you are offline, or why delivery of their product will be late, or that their data has been compromised.
- For Advanced Persistent Threats, consider adding a step in your recovery procedures to scrub your back-up data for malware, too.
Finally, and this is an important step for success, these plans should be exercised both independently and in concert with each other. What if a cyber-attack takes down the power? What if significant numbers of your employees are hit with crypto-ware and rendered unable to perform their regular job functions? What if the malware code is present in your backups, too?
Any gaps discovered during testing must be raised to the appropriate level, included in any risk tracking initiatives, and remediated quickly. Third-party assessments can also be used to help validate plans and procedures in accordance with various standards or frameworks such as Federal Continuity Directives, the Continuity Preparedness Guide from FEMA, National Institutes of Standards and Technology special publication 800-34, ISO 23001, or the Federal Financial Institution Examination Council.
5. Emerson Network Power, 2016 Cost of Data Center Outages Report.
Latest posts by Eliot Schmidt (see all)
- BCDR Series: The intersection of Incident Response and Business Continuity - October 6, 2016
- BCDR Series:Retail Resilience – Retailers Suffer Major Outages Ahead of Holiday Season - December 7, 2015
- Is Your Business Ready for the Storm after the Storm? - October 1, 2015