One of a client’s bigger pain points is dealing with backups. My task is to define the correct solution for the client at cost that they are comfortable with. I have a standard list of questions when I’m designing a solution. Here are a few of them:
- How much data can you afford to lose before it affects your business?
- How long can your network be down before it impacts your profits?
- If you come into the office tomorrow morning, and your server and backup drives are missing, can you recover?
- Do you have any mandatory regulations on how long you must keep your data?
- Do your backups need to be encrypted?
- If you are using external hard drives how often do you change them and take them off premise?
- Have often do you test your backups?
The answers provide me a baseline for developing the client’s backup solution but my topic today deals specifically with item 7.
How often do you test your backups?
Many of the current products will take a backup of the computer, then once a day will test the backup by booting into a virtual environment and confirming it recognizes a logon screen. Most will even email a screenshot of the logon screen to you. This is a great advancement in the technology. Sadly this can also provide a false sense of security.
Have you ever had a computer show the logon screen, but when you logged in it gave you the Blue Screen of Death (BSOD)? This can happen with backup products as well. The only way to confirm your backups are valid is to launch them into a testing environment.
Recently, Mirazon received a call from a client who contacts us only when they have an issue, which is infrequent. Their small business server was down. When the engineer arrived on site he was pretty confident he could restore the server in short order as the client has a highly rated backup solution for virtual servers.
When he finished the restore, he booted the server and it came up to a log on screen and immediately rebooted again, and again and again. After a number of hours online with the backup solution’s support and then more than six hours online with Microsoft, the only solution was to rebuild the entire domain from scratch and then restore the only data files that were still accessible. So, not only did this service call require a server rebuild, but every client had to be rejoined to the network.
Not a happy process. But, like most disasters, it could have been avoided. Scheduling just an hour a month with an engineer to fully test of all of the backups would have resulted in saving the client from being down for days, a basket full of aggravation and a very large invoice.
Backups are a good thing but, like a famous politician once said, “trust but verify.”