The Big Boss of Initechs Australian division ran the Sydney office as his own personal kingdom. Work- or workers- he didnt care for was banished to the hinterlands of the Melbourne office. For example, IT services was a useless sack of morons who only know how to spend money, and thus the entire department was banished to Melbourne.

Stewart C. lived in Melbourne, and was a new hire not long after the exile. The Melbourne office, with a 900km buffer zone protecting it from the whims of upper management, was actually a decent place to work. At least, until Brendan arrived.

Brendan was an upwardly mobile middle manager based out of Sydney. One day, he did something to offend Tony. The scuttlebutt had a lot of rumors about what that was, ranging from oh, he was late to a big meeting, to he took the last beer at the company party. Rumors aside, the facts were clear: Brendan was in Melbourne with a long list of projects to manage, and he wasnt going back to Sydney until he completed all of them.

A few days into his exile, Brendan brought that list of tasks to Stewarts cube. Stewart. Stew. Stewie. Im gonna need you to go ahead and test our backups, yeah? Make sure we can actually restore from them, right?

Thats a great idea, Stewart said. So great, that we test them on a regular basis. I can send you a doc that covers our backup strategy-

Right, yeah, but have you tested them in production? Brendan asked.

Stewart paused, making certain that hed heard the question properly. Well, no…

Right then, yeah? They havent really been tested till we run them against production, yeah?

We… really shouldnt test things in production, Stewart said. I mean, what if the test fails?

Brendan shook his head and laughed. Stewie, why would the test fail unless the backups are broken, right? Yeah, Im gonna need a plan to cycle through every production server. Wipe it, restore from backup tape, and confirm its working, yeah? Right, so get on that. Orders from Tony, yeah, we need this by next week.

Stewart immediately got to work- on making sure this hare-brained idea didnt go forward. He roped in his boss, who roped in her boss, and everyone was on the same page: this was a terrible idea that had no real benefit and carried with it too much risk and downtime. Buuuuuuut, his boss said, its what Tony wants. We havent got the political clout to tell him no, so were going to have to do it.

Wind of Stewarts attempts at killing the project got back to Brendan. This resulted in a rather nasty meeting invite demanding a status update. Stewart wasnt the only invitee. When he took a seat next to Brendan, they were both staring at a speakerphone.

The phone resonated with Tonys booming voice and confident authority. Well? Whats all this then? he demanded.

Ill tell you what this is, Brendan said. Its just like you said, sir- everyone out here in Melbourne is lazy and obstructive, right? Yeah.

Youre telling me crap I already know, Tony said, his voice crackling on a bad connection. Of course theyre useless, thats why theyre out in Melbourne. Was there a point to this call?

Well, yeah, right, Brendan said. I wanted you on the line for this because Stewart has been the worst, right? Hes been nothing but a trouble-maker disrupting my efforts and second-guessing me-

Stewart started to mount a defense, but he couldnt get a word in before Tonys laughter drowned him out. Brendan, if you cant handle the Melbourne office, how do you expect to cut it back here in Sydney? You just havent got the guts for this business, Brendan. Now, cmon then, youve got your orders. Chop chop, and no excuses.

Tony ended the call, leaving Brendan and Stewart staring at each other in uncomfortable silence. Brendan was the first to break the silence. Yeah, as you can see, Brendan said, this project has attention right from the top. So youre going to give me a workable plan for testing all of the backups on the production servers by tomorrow- or else, right? Yeah.

Stewart nearly demanded, Or else what?, but decided against it. Instead, he wrote up his plan, noted the steps that involved downtime, and sent it to Brendan the following day. Brendan didnt have any follow up questions, and nobody told Stewart to execute the plan, so he promptly forgot about it and got back to doing real work.

Until all IT operations came screeching to a halt two weeks later. Brendan, as it turns out, decided he didnt trust Stewart to run the tests. Instead, he strong-armed one of the junior technicians into doing it for him- but not until after making his own modifications to Stewarts plan. Specifically, he wasnt happy with the schedule- Stewart had very conservatively scheduled only one server to be tested a week, outside of regular business hours, so that any failures or issues would be minimized. That would have taken months, and Brendan wasnt willing to wait that long, so he made the junior technician do them all at once. Without coordinating downtime.

The result was every bit as catastrophic as one might expect, especially considering the junior technician didnt think to order the backup tapes before starting the test, which meant they needed to be shipped from their offsite location. All in all, they lost two days of operations and a lot of reputation among their customers. Tony was furious, and he knew exactly who to blame: the useless sack of morons who only know how to spend money.

Obviously, the folks in Melbourne couldnt handle the complicated job of running IT services, so the IT department there was closed down at the end of the quarter and Initech hired on contractors to set up a new datacenter in Sydney. Stewart wasnt particularly happy to lose his job, but he took some solace in the fact that he had prospects lined up- while Brendan didnt.

http://thedailywtf.com/articles/tested-backups