If information technology is your career, you have experienced or will experience an IT emergency or crisis at some point. If you have not, it is only a matter of time.
An oft-overlooked aspect of operating during an IT crisis is communication. Without proper communication, whatever crisis you are dealing with will descend into chaos: no one on the IT team knows what everyone else is working on; people are all trying to do their individual tasks and conflicting with one another; users get frustrated; management does not have answers for their leadership; everyone is angry. Cats and dogs living together, mass hysteria.
A proper communication strategy is key to keeping the wheels on during a crisis. For the purposes of this article, a “crisis” is an all and/or critical systems down scenario. This can be a ransomware infection, a total SAN meltdown, a core network failure, basically anything that takes the business offline or severely degrades its ability to operate.
Have a Plan
I’m not a nerd or anything (editor’s note: yes, he is), but I am going to talk about comic books. It is often said that Batman has no “superpowers.” I beg to differ. Batman’s superpower is contingency planning. A common theme when discussing the caped crusader’s chances against other comic book heroes is, “how many days of prep?” It was revealed in JLA: Tower of Babel #14, that Batman has done full analysis and has a plan to take down each one of his teammates. Be the Batman of your IT department, have a DR plan.
Just because we like you, here’s a link to our template to start your DR planning.
Declaring a crisis or determining a level of disaster can vary from organization to organization. Define your scenarios ahead of time. The DR planning template we provided allows you to do this and break it down by system or business service.
Communication is a crucial part of your DR plan. Do not try to figure out how to properly communicate in the middle of a crisis. In the words of the highly esteemed songified Internet Meme Lady, “Ain’t nobody got time for that!”
Three Lanes of Crisis Communication
When dealing with a crisis, there are three potential lanes of communication. Each will need to be handled in different ways:
- Internal technical communication. This is the communication amongst you, your peers and any trusted advisors (like your friendly neighborhood Mirazon technicians) that are on your team. This is going to be a deep technical conversation. In this lane you will work together to diagnose and discuss solutions and divvy up remediation tasks.
- Vendor communication. This lane is like the internal technical communication. You may or may not merge this with the internal technical conversation. You will be working with vendor support for a fix, arranging RMA’s and so on.
- Communication to the business. This lane of communication exists solely to keep management and end users apprised of progress and new developments. It is not a deep technical conversation. Keep it clear, concise and free of “IT speak.” For example, trade, “The TCP/IP ephemeral ports in Windows’s TCP stack are causing latency above 70ms to the SAN” for, “Looks like there is a bug in Windows causing issues connecting to the SAN.” You get the point.
Have a Designated Communicator/Spokesperson
During a crisis, your team will need to be able to focus on the issue at hand. There will be little time to explain what they are working on or what they have done to several different people. Their attention is required on the issues, not dealing with interruptions and context shifts. Appoint someone to be the “communications hub” for the issue.
This person is responsible for getting updates from the technical group to the business and vice versa. The designated communicator acts as a buffer between the people fixing the problem and the people impacted by the problem. This person will be a bridge between the technical team and the non-technical decision makers so they must be well equipped in communicating to the layman and able to handle being questioned.
Know your team’s soft skills ahead of time so you’re ready to make this call quickly and efficiently.
Be Clear, Concise & Consistent in your Communication
It can be tempting to get “flowery” with your wording, maybe to try and do justice for the work your time is doing or to try and look like you’ve got it together when you may not...but now is not the time for that. Avoid any of the corporate buzzwords or jargon. No one really knows what that stuff means anyway. You did not suffer a lapse in your high availability paradigm. High availability did not work, you will need to investigate after things are up.
Use a regular communication cadence and your updates should be given with consistency. Decide what that is and what medium you use (email, phone call, Teams or Slack) with your management and stick to it. Set a timer to go off as a reminder. Giving an update that there is not update is better than no update at all.
Stick to the facts, be clear in your language. If you have a guess or hunch about something and you mention it, emphasize that it is just that. It’s a hunch. If (when!) you are writing your communication, use clear formatting.
This is what happened:
- This happened
- That happened
This is what we have done:
- We did this and it worked
- We did that and it did not work
These are our next steps:
- We are going to do this
- We are going to do that
That’s it. Simple, easy to read, no fat.
Reiterate Your Communications
People tend to hear what they want to hear. I cannot count the number of times I have been on a call where everyone walked away having “heard” different things. For this reason, back up each verbal update with a written update via email (preferred), text or IM.
Remember, YOU are the Experts
You are a paid, career, IT professional. It is your duty to provide good information. That information may not always be what management wants to hear (“We need to shut down all day?! Are you insane?!”). You are not doing anyone any good if you tap dance around an issue. Don’t try to avoid upsetting management for the sake of not upsetting management, but management needs the facts so they can make good decisions.
Set realistic expectations. You will regret committing to unrealistic timelines and solutions you do not think will work.
Remember, management pushed to open Jurassic Park before IT was ready.
Using these communication strategies paired with a strong team, you will make certain your next crisis only sucks and not double sucks. If you need a tiger team to come in and help you wrangle your DR planning or assist during a crisis, we’re here to help.