So I had the privilege of moving offices recently. In fact, December has pretty much been dedicated to getting moving. So where shall I begin …
We should have moved into Duncan 1061 (The big white building next o computer corporation). This didn’t quite work out … building contractor couldn’t clear out fast enough. So beginning of the month we re-initiated the hunt for new offices along with Interexcel, SABest and Thinktank. What a mission. We spoke with hordes of agents, looked at what felt like hundreds of different offices (all whilst still maintaining normal work/performance levels – read: EXTREMELY LATE NIGHTS). After a day or so we settled on an office in “The Village office park” on the corner of Glenwood and Oberon in Faerie Glen. Ok, so now it’s re-contact every supplier making sure that they’ve got the right addresses, recalculate the network point locations, and requirements (as we’ve lost Interexcel Training along the way). Ok, so at least this brings the network requirements down by one switch and about 10 network points (still enough to clear out 3 patch panels (24 ports per panel) throughout the building, and another on in-server room network points (well, 18 in the rack and 6 in the server rack, with a 24-pair cable running into the VoIP server, of which only 12 is currently populated, 8 FXO – Telkom facing – and 4 FXS – facing towards FAX and ather analog equipment only), with three ADSL lines (this is partly where the large number of server rack patches is coming from, 5 servers, two of which has links to a diginet router and three DSL modems, and four internal networks, adding up to a required 12 points in the rack – we have 6 extra since that is what the patch panels allowed for and we’re already looking at adding another two possibly three servers).
Ok, so Monday the 8th of December. I get news in the morning that they want to start putting in carpets by the 10th (they’re deadline for sign-over is the morning of the 17th) and I should get my cables in before then … I finally get 1km (should actually have gone for 2km but I didn’t have sufficient time to make the calculations) worth of CAT5 by 16:00 and is ready to start pulling cable by 17:00. By 18:30 Nuaghty Systems CEO, his wife and two workers pitch up to help (by this time I already ripped out close on 1.5km worth of cable from the existing infrastructure which was terminating at the wrong locations so we couldn’t simply re-use). So we spend the night (we walked out of there 5:30 pulling in around 2km worth of cable, 1km of mine, about 300m of Nuaghty’s and a LOT of scraps we’ve salvaged). This is cabling only as the swing frame I received earlier was incorrectly coloured (and I had no energy left to still wall-mount it – walls still needed to be painted as well).
So after Monday being a day from hell (it felt like _everything_ broke) the evening was a through-nighter. You’d reckon I’d manage to catch some shut-eye, but as I’m the only sucker with keys I need to be back on site at 6:30 so the workers can get back in … so just enough time to grab a shower to get rid of the “think pink” fibre glass crap. I hate this stuff with a passion btw, in case the previous sentence didn’t quite get the point home.
Tuesday I should have spent on site putting in wall boxes, however, Tuesday turns out to be another day from hell. Partially due to me being overly tired and partially due to me starting my fight with the electrician that seems to be doing whatever the heck he wants (eg, not putting in the ordered dedicated “red” plugs, putting the “poles” whereever he thinks they should go – we never ok’ed the locations and these were still to be specificied – putting the power plugs at uneven hights on the poles, and cutting the cover strips at these heights, giving me bullshit directions to the place since he can’t explain to me what the network boxes looks like, putting the power boxes on the poles at > 700mm heights causing them to overlap with desk surfaces – popular theory states that two solid objects cannot occupy the same space-time, screwing up half the neutrals causing the power to randomly trip at various odd times). Damn, I realized the guy was useless, but putting some of the stuff down in writing just made me realize how badly. Reckon the fact that breaking bigger holes than what face-plates would cover into dry-wall, punching 30mm holes where a 6mm hole is required and generally being an arrogant (fat) … I would use kaffir but that would get me labeled a racist (and this isn’t aimed at race, it’s aimed at incompetence). Even Jason got fed up with this guy.
The red plugs in the office space got ditched as getting the guy to fix this would take too much time (remember we’re at Wednesday), however, now he didn’t get it done in the server room either (where we clearly stated we still wanted – but I’ll forgive him for this since Jason too misunderstood). Anyway, I’m knocked out of action until Saturday due to the carpeting and I’m not allowed on the carpets for two days.
Saturday morning I get in there and what a cock-up. Half the plugs aren’t working. At least three DB switches would trip when pushing them up (with no locatable load). The lights would cause the DB to trip at irregular (max 5 minute) intervals on yet another circuit breaker. The poles look like the gremlins got hold of them (I told the electrician to his face that it looks like an ape did most of the work … I doubt he realized what I implied). The (relatively) carefully organized network cables got batched up and thrown into the roof (I’m not sure who to blame on that one … I just cable-tied the cables into looms according to direction/termination area and rolled them up so that they won’t lie on the floor … somehow they STILL got messed up). Saturday I spent most of the day doing stuff that I can’t even remember (probably mostly argueing with the electrician).
Sunday I took off. Bliss.
Monday it’s back to work, at least now our offices officially closed, however, I chase around a part of the morning after some stuff, in the afternoon we (at least now the people working for me was also on-site) quickly punch in the network cover plates for the poles and put down the wallboxes, and mount the network cabinet. We spend a lot of time moving stuff from the old office to the new one as well, thus losing a lot of work time. By 18:30 we decide enough is enough and since the electrician and aircon people are occupying our server room with lots of their garbage continuing is difficult anyway.
I meet Jason on-site on Tuesday (the 16th). The electrician finally pitches up but once more manages to slip out. By now I strip my moer pretty badly (he promised me he’d pull in some network cables into the drywall for me, which he did, but he punched 2 x 30-40mm holes at each of the locations, in the wrong locations as well. Needless to say – this rather irritated me, so I just removed his plugs, put my hand into the dry-wall, punched a small 6mm hole at the locations where I originaly indicated to the clown that I wanted the cables and pushed them through. Jason wasn’t particularly pleased that he had to fix the walls…
Fortunately we also managed to start punching the network cables into the network cabinet. So things started to look slightly better. However, if you’ve ever done any kind of cabling, you know how much cable pieces comes off from crimping/kroning a single cable. Imagine doing 60, on a carpet (which was supposed to be non-static tiles). Whilst I’m on the topic of tiles vs carpets. anti-static _tiles_ serves two purposes, one, it’s not supposed to build up electric charge, and two, it repels dust, resulting in a supposedly cleaner environment. Carpets (no matter how much anti-static glue you rub into it) will always attract dust like manure does flies. So after the room went through 3 telkom technicians, an aircon installer (which is STILL not installed), an electrician who works like a baboon and myself (the ground on top of the roof panels and inside the “think pink” stuff didn’t help either) this place is a mess of epedermic proportions (read, you can’t step anywhere without stepping in visible dust). After numerous fights with the electrician about our plugs (he did at least assist with grinding the trunking and roof-struts for T’ing the trunking and getting into the roof – and not mess it up badly) not being right I’m still not willing to move electronic equipment into the room (no windows, single door, extreme heat and incorrect plugs), so by Friday finally the plugs are red, on a seperate circuit and two of the five actually works. Just a shame I can stick my finger behind some of them due to the cover plates being bent like bender. Not safe. I still refuse to move in, but I do finish all kroning … and wall-mounting of equipment by Saturday afternoon.
Now, in terms of suppliers, I’m still short one small strip of trunking (Suppliers out of stock). The people supposed to be providing our chassis provides us with 450mm rail kits (with servers that are 550mm on chassis depth which won’t fit a 600mm cab, associated with 450mm post front-to-back post distance, due to the handles and cable extrusions at the back taking it to about 650mm). So I pick a fight with a friend, and vow to never use that particular supplier again (three reasons: 1. They supplied obviously wrong equipment, 2. I like my friends more than cheaper prices, and 3. they retrenched some other friends of mine). So I drive off to their supplier’s suppliers due to seemingly nobody having drivers available, 4 chassis in a polo playa seeking for 650mm rail kits for the chassis. Turns out IRS is the importer (albeit, I suspect the rails are made locally) and I get what I want. I also get to learn that not only did my supplier ship wrong rails (and they were short one chassis on my “lower spec” chassis), they also shipped me the really crappy quality, and more expensive, 4U chassis for the “lower spec” chassis (ie, can only handle 3 x 3.5″ drives). So I phone them and negotiate a return on unspecified grounds (Rudolf, if you read this, it wasn’t just the rails … I could probably punch a hole through those chassis with my pinky, and the fact that I really needed that extra chassis more urgently than I let you guys know), buy three chassis (much better quality) direct from IRS (at about 40 % less). So now I have 7 chassis with me, two of which needs to be returned. I should post the two photos I took with my phone … it’s amazing but I actually managed to squeze them in.
So one would reckon that this would be ok now … well, as it turns out the rails don’t exactly fit on the chassis. The holes are mis-aligned unless you put the rails way too far forward (by removing the chassis handles), and they also end up putting the chassis at a 2/3rd U boundary (meaning that I waste 1U of rack space for every “set” of servers), and causes the front-handles to mis-align so I can’t fixate the chassis against the rack posts … anyway, the fixing the rails to the side of the chassis problem was solved with a grinder. Yes, the type that cuts into metal, causes stuff to heat up and in general cause a lot of discomfort when those “shards of metal” comes flying onto arms and hands (I could feel them burning through my jeans … they really are quite hot … instant tan).
Ok, so finally something goes right – we actually manage to fit in all the power, kvm and network cables into the rack without any problems (they actually fit into the posts, so there are not more cabling exposed than that which is absolutely required … again, need to post some photos). Ok, so what remains for Monday is for the electrician to install power for the aircon, and for the aircon to be installed. We’re now at the 22nd. neither the electrician, nor the aircon people pitched today. According to Marius the official status on the aircons are “they’re lost – nobody knows where they are”. Either way, Quinting and myself spent the majority of the day re-building the last of our servers into the new chassis (previously we had some standard desktop chassis that was standing on the floor, consuming lots of space and generally being a nuisance to work with.
At 13:00 we decide we can’t wait any longer and that the aircon guy is just going to need to be briefed properly about not causing a mess (I spent about 2 hours with a vacuum cleaner today cleaning out carpets) and we start removing servers from the rack so that we can move it into the server room. This is done at about 16:00 if I’m not mistaken and we start pulling in the last network cables, and telephone lines into the rack from the various patch panels and to the in-rack patch panel. Well, it’s at this time that I realize my punch-down is MIA (and I’m itching like flippen “think pink”). So we do what we can to at least pull in the cables, and I’ve got a picture here somewhere that shows the mess the lack of a punch-down can cause.
So physically the work that remains is to actually punch the server-rack network and telephone cables into their appropriate locations, locate some rack-mounting brackets for a CISCO-2600 and a 19″ LG 16-port rack-mountable switch, need to stack the DSL modems and punch them into the patch panels. If I had a punch this would be about an hours work. As it stands … I don’t even want to think about it. The last I saw my punch-down was Saturday … in the server room where I finished the last network points … or by the rack where I thought I might need it but ended up only doing the RJ45 side of things. Anyway … I’m suspect of the Telkom technician, it’s the only guy that was there in the last week that could possibly have motive (Or Cassie/Kevin/Wayne/Stephen trying to prank me – bad timing guys … I’m a tad on edge). Frankly, I think it’s just misplaced. Along with my screw driver (last I was this was when we were assembling chassis on Saturday … what’s the chance that this ended up inside one of the servers?!?). Then there are also some trunking that needs to be re-fitted. This shouldn’t take longer than 30 minutes … but I will need Zuma back on site as he did miss a significant amount of touch-up paint on the trunking. Then at least the big pain in my backside is done.
Configuration wise there (fortunately) aren’t much to be done. Some reconfiguration of IPs, updating of internal DNS and updating of VPN rules. The managed switches also still needs some partitioning, but that will wait until we’ve managed to complete the monthly invoicing … which I’m not sure how we’re going to push out the mail yet … since I can’t get the DSL up without a punch-down and the 3G … well, let’s just say pushing out a thousand emails over 3G at 100KB/email is not only going to take a while – it’s also a heck of a lot more expensive than simply making use of a local-only DSL account. Anyway, if I can complete the above physical stuff I should be able to configure the rest of the critical systems in about two hours (not difficult, just meticulous work).
No wait, there is one other nightmare that still plagues me:
Telkom installed 8 lines, with absolutely no indication of which lines belongs to which companies, which are normal analog, what the hunting configurations are, or which ones are DSL and which are not! So … if anybody has some friends at Telkom … please do make the introductions – or it’s me and that highly annoying “Thank you for calling Telkom business, the network that makes business work” … I’m so sick of that it makes me want to puke just thinking about it. As mentioned in other rants – my techies spend more time on the phone to Telkom than to any other single entity in existence.
Oh well, time for bed. Tomorrow is still a longish day ahead …