Over my years I have spent considerable time in server rooms, from multi air-conditioned custom designed environments, large data centres, glorified cupboards, and basements. Throughout that time I have seen some interesting sites, from sleeping colleagues, a panic stricken IT engineer
holding up a stuck server half out the rack and more than once interrupted some embarrassed staff having extra office relations (archive rooms adjacent to server rooms)…but every single one of them pales in comparison to the server room in a toilet…yep you read that right…a server room in a toilet. The mind truly boggles at what cretinous imbecile(s) decided that the best location to put their business-critical server environment was the same room as a significant amount of plumbing.
Working for a large Oil company it was not uncommon for smaller companies to be acquired, the process was standard, install our kit, migrate data\apps\users, remove their kit. Commonly we would send an installation engineer to carry out the physical installation of their new kit, however as this company was based in Aberdeen which was local to myself so I was given that honour. I headed off with the various kit, the usual tools and a contact name at site, after I arrived and went through the usual health and safety routine (waivers, fire assembly points etc) my contact asked me to follow them. Off up the stairs…”is there lifts?” I asked shuddering at the thought of lifting all that kit upstairs…thankfully there was…phew. At the second flight there was a swift left turn and my contact walked in to the gent’s toilet…I figured having a quick comfort break, so I waited outside…30 seconds later he pops his head out…are you not coming in? Seriously perplexed I was about to query if he thought I was a plumber when he said, “server room is just in here” and motioned to a second door in the toilet…I had a quick glance to see if there were some cameras recording for a bit of fun at the IT geeks expense. Alas, no such luck here lieth the server rack, along with numerous flammable cleaning chemicals, cloths, spare toilet roll and 6 boxes filled with paper archives, “well at least we have a water source close by should there ever be a fire in here” I said with a modicum of sarcasm…clearly lost on my contact who replied “that’s a great point”, I cringed and shed a little tear to myself, then with a “good luck” off my contact toddled.
First task of course was to review the existing rack, now where’s the light switch…of course there is not one. My only option was to leave the door open, so I at least had some light in the server room but round the back of the rack you could barely see a thing, so my trusted phone torch came out. I immediately encountered the first issue, it was a tight squeeze to get round the back of the rack and I am your usual skinny geek, issue two was then identified it genuinely looked as if they had got a big old pile of various cables and threw them at the rack to see what stuck, trip hazard galore (remember that scene in Indiana Jones with the pit of snakes?). Surely this cannot get any worse, to then find there was not enough space at the front of the rack to extend the rails for removing or installing servers…truly one of those points in an IT Career when you seriously question your life choices. Deciding I needed to get a new plan of action I mentioned to my contact it will take a good bit longer as it is far from a great environment, have you ever had the urge to garrotte someone with a network cable? I came close when he chuckled “yeah it is isn’t it”
I grabbed a spare desk and came up with a new plan and with new found conviction I headed off in to the toilet, literally as I entered I was greeted with a stench so rotten I genuinely thought something had died, then a flush was heard and a lad walked out the cubical went rather red and proclaimed “sorry about that”. Then for the remainder of the day there was a stream (pardon the pun) of people in and out the toilet (1s and 2s), I am still convinced the lot of them spent the week holding it in and had ample servings of prunes for breakfast. After several hours of techniques that would amaze a yoga master I rearranged the rack, got the new kit in and all connected…I called my boss to advise that the job had been “Sh*te” to which he howled in laughter after I told him why, what’s even more alarming I still don’t classify that as the most “Sh*te” job I have ever had.