Tales From The Field: Always Have A Backup

I’ve had a lot of people tell me they have backups but have never really needed them. That they thought backups were unnecessary. Some have said they were going to discontinue them. When I think about this I first find it amazing that they have not needed their backups and hope they are someday thankful of that. I think on some horror stories I know where years of good content were lost because of no backups or even cases where backup corruption attacks. If you’re on the fence about backups consider a story like this could happen to you.

For the record I’m going to keep these stories anonymous. While I think these stories are useful to help us see why we would want backups I’m not going to name my friends and cohorts for the bad or painful things that have happened to them.

Years of Blog Posts Lost

A friend of mine used to host his blog with a shared hosting provider. The hosting provider listed on their site that they did backups of the content and it was included in a package. After being with the hosting provider for a couple years they had a catastrophic outage. The codebase and the database were lost. You’d think the codebase (an open source project) would be simple to reconstruct and the host had backups of the database - which wasn’t just marketing BS. But, this outage also corrupted the backups. It was all lost.

This was a lesson for me in having backups separate from your service provider.

A Config Glitch Corrupts A Database

Imagine a site that uses multiple servers and is hosted in a cluster. Deployment management is all nicely scripted. One day a server config update is deployed and it all goes wrong. A regular running script does something that corrupts a database. It was a race condition that would only show up in an outage. Now the site is down. To make matters worse a recent update to the backup scripts caused the compression of the backups to generate corrupt files and no one was the wiser. The content is corrupt.

This case didn’t turn out entirely bad. There was a second set of backups that fired just at deployments and updates in an automated manner. These were used to bring the site back up and visitors were basically unaffected.

The lessons I took away from this story are to test your backups semi-regularly and to create backups right before any kind of update that you test before doing the update.

Bye bye Family Photos

Backing up web stuff isn’t the only smart thing to do. For example, I know someone who only had their family photos on their laptop. It was a computer used everyday and it didn’t seem to have any problems. In fact, no computer this person had owned for the past couple decades had ever had a problem. Then one day part of the hard drive started to go corrupt. It went bad before they realized what happened. Loosing your music and documents is annoying. Loosing photos of your kids is something different all together. I know several people something like this has happened to.

What I took away from this was to make sure my personal stuff was backed up.

Next Steps

After writing this I realized it might be useful to share some of the backup strategies I’ve used or am aware of. Look for a follow-up post with some simple actions people can take to start backing up their stuff.

Update: The post with backup solutions is up.