Week 2 of our new network migration
by, 1st August 2012 at 07:45 PM (11752 Views)
And things are looking surprisingly well. Few stumbling blocks but nothing to write home about.
First week was always going to be a bit iffy as there were still a lot of staff in. But a great opportunity to build the majority of the ICT suites and break the back of the heavy moving.
But why, why oh why did we have to do that in the one week it's constantly 30ºC?
Despite the email updates and reminders to staff we still had one member of SLT complain when I reminded everyone that Friday was D-Day for the old network. Held off until Sunday night after spending all weekend remoting in (well, remoting in for 5 minutes, starting a robocopy for one batch of users then leaving it for an hour or so) copying all users off the 3 servers onto the new namespace, followed by RMPublic, RMShared Documents and RMStaff.
It's astonishing how much data piles up. Data in the above shared locations and all users My Documents tallied 900GB. That's a fair whack considering we have a special NAS for large media, images etc, but a quick run over it with Treesize Free shows there's still a lot of people not making correct use of those facilities. If anyone has a handy way of blocking a combination of certain file types over a certain size (for example .avi over 400mb) from home folders then I'm all ears
So we tallied 900GB of current data. 2.3 million files. Wowsers.
Monday rolls around and I'm itching to turn off the old servers but we needed to play it safe and check it over several times - sure enough we found a few other resources we will/may need so copied those over. Tuesday morning, the bullet was bitten and all 3 RM CC3 servers were permanently removed.
This proved to have a particularly positive effect on one thing at least. Our NM came in not long after I had done this and put the new server in it's place.
CC3 Server 1 has MASSIVELY loud power supply fans.
NM sits RIGHT next to the server/switch cabinet and we've often wondered how we can convince people to move either it or us due to the high level of noise. We'd previously pulled some favours to get a decible reading done and it fell about 5db below the threshold for H&S to play a part, however no doubt there would be differences due to it being constant and right next to him. But we had other things on our minds and let it be. With this server decomissioned, the noise difference is MASSIVE - we can hear the phone clearly, we can have the radio on at a volume that won't annoy the people in the next town and we can hold a conversation without sounding like a pair of 80 year olds.
What? Speak up young man...
As mentioned in a previous blog post, our intention was to re-use the chassis from our most recent RM CC3 server (a high quality Chenbro unit with redundant PSU and a Sata/SAS backplane in place) to house the second new server gubbins. The above noise issue won't be a problem as it's being housed at our other site in an unmanned server room.
All went reasonably well - fitting was easy, but the chassis had some welded in standoffs that sit in the wrong place. Just 2. One isn't a problem, there's nothing on the motherboard that it interferes with, but the other sits right underneath the second batch of RAM slots and even when covered over with plastic tape, it renders 2 RAM slots unusable. So down to 20GB RAM. If it causes a major issue we'll have to rebuild it after yanking the damn thing off but for now we're not too worried.
Secondly, the power supply being slightly older only had a single EPS12V power cable. Bugger - can't run dual CPU with only one. Searching for adapters online came up reasonably fruitless - adapters either converted a 4pin ATX and 4pin Molex to 8pin EPS, or some very dodgy looking single molex to EPS. The current flowing down those things, I'm not taking the risk. So for the time being, we thought we'd just build it with half the RAM and single CPU to make sure it works OK. Not a problem - just pop the plastic protective cover back onto the empty CPU socket and F***! Slip of the finger and I've bent 2 or 3 of the pins on the CPU socket. Big swear words. I'm glad no kids were present because both F and C bombs made frequent occurrences.
With the help of NM and having the board removed again from the chassis, a steady hand with a phone-torch, magnifying glass and a jewellery screwdriver, we managed to get it looking vaguely straight. Cleaning bill for the pants, please!
Later, revisiting the EPS thing, I made my own adapter - twin molex to EPS with suitably large gauge wire. Works a treat, and I trust my own soldering skills better than some of those cheap cables from amazon! Thank god for the army of spare cables I hoard at home
Wonderful. No going back now, the old server is dust. Almost literally - blowing it out before putting new stuff in it caused a dust cloud of epic proportions. I did it in the hall and the guys chopping up rainforest amounts of wood in the room nearby wondered what the hell had gone off.
Server built. Hard drives installed. ESXi installed.
Unreliable though - vSphere kept flaking out and I couldn't keep it up long enough to warrant copying over the new VM's to it. Bugger - I hope that standoff isn't causing more problems than initially appeared. Maybe it's the 4 port trunk. All cables out - no, still pings fine. Hang on - all cables out - still pinging. Gack, something's stolen my IP address!
5 minutes later and a rogue NAS moved to it's correct IP, problem solved.
Domain Controller 2 moved. Powered on. Success!
SCCM Server 2 moved. Fail.
I had both the above VM's on the initial new server along with the other 2, and I had set up DFS-R to replicate all users and all shared directories. Filled up the datastore damn quick. I didn't worry too much, it'll run as well with 1 server as it does with 2. But trying to move it across to the new server with plenty of space, it was clear the 2 larger virtual drives were corrupt. Thankfully it's no big deal, in interest of keeping things as modular as possible I had set up 3 virtual HDD's for each server instead of partitioning one. So OS could be brought up and the other two remade within 5 minutes. Replication is currently taking care of the users and shared stuff, and just re-setup the server in SCCM on server 1 to reinstall the site server gubbins, distribution and software update points etc.
In the meantime, my colleagues were making considerable headway into building/replacing/upgrading the 450 odd stations around the school, almost entirely unaffected by my server moves and tweakery. That makes me very happy, that's failover working exactly as it should. And of course moving the plethora of other servers and services to the new domain - terminal server, Moodle, email servers, websites etc.
So from now on, we just need to concentrate on the "little" bits.
Gettiing GPOs and security just right.
Printer deployment. Still need to figure out best practice on that. Many people have many methods. Some have a single GPO with all printers with massive security filters so the right machines get the right printers. Some have a GPO per location, some have a GPO per printer.
The latter method seems to suit us best as our locations in AD aren't always sharing printers. So I think that's just make an empty GPO for each printer, lob in the correct machine/location assignments and use Printer Management in 2008R2 to deploy to those GPOs.
Software packages and deployment.
There's some big buggers in there. Solidworks made me grind my teeth on CC3 however I seem to recall last year that doing the same on vanilla was considerably better documented. Adobe CS5.
SIMS (for future use - currently we've a separate image with SIMS installed already).
There will always be a few odd ones. Most the little stuff we have in our base image like VLC, Irfanview, Office etc. But I'm happy enough with that. And if there's anything too awkward we can always just throw it out via Impero MSI deploy
So, things are looking positive. We're on schedule, there's been a few swear words uttered but things are working as expected. My ultimate aim is that we can have some time off at the back end of the holidays without feeling worried about anything. We only do half days in the summer as it is but 2 out of 3 of us have kids and would love to spend just that bit more time with them whilst the weather is still good,ish!
Total Trackbacks 0