Yay! It works!

[dhicks]

Dead chuffed with my replicated virtual machine image setup - main file server went bang (literally, complete with large electric-green flash from deep inside server, cartoon-esque "fzzzzzt!" sound and, I am assured, a lingering burnt smell), but the replicated virtual machine image was able to take over in short time.

 

My cunningly-crafted fail-over scheme ambled rather than swung into action as it turned out that said server explosion fused the plug to the UPS that 3 other servers were plugged into, so I had to sort that out first, but everything was back up in half an hour and some users didn't even notice the server was down.

 

Now all I have to do is figure out how to get the server working again. Haven't dared plug it back in again yet, I figure I'd better wait until the end of school in case it trips the power supply...

 

--

David Hicks

[chrbb]

Any details on how you set the virtual image up?

[maniac]

Nice one, always good to know those hours planning disaster recovery are worth it. Make sure a full report is filed with your SLT as unless they are told, they probably won't have a clue that it's all down to your hard work the school kept running as usual.

 

The school I work at recently had major problems with it's main DC, and as a result I'm told almost a weeks worth of down time, totally un-acceptable in this day and age! This was before I started, so yet another thing on the agenda in the next few months, write and implement disaster recovery policy!

 

Mike.

[dhicks]

Make sure a full report is filed with your SLT as unless they are told, they probably won't have a clue that it's all down to your hard work the school kept running as usual.

 

I shall present the offending power supply, possibly with a small bow on top, to the bursar in the morning.

 

--

David Hicks

[dhicks]

Any details on how you set the virtual image up?

 

Virtual machines running on Xen on CentOS, mirrored between machines via DRBD. Nice, works a treat, even on our (as you can tell) cheaper end of the market hardware.

 

--

David Hicks

[azrael78]

Virtual machines running on Xen on CentOS, mirrored between machines via DRBD. Nice, works a treat, even on our (as you can tell) cheaper end of the market hardware.

 

--

David Hicks

 

Manglement here isn't a fan of Linux et all - but if it saves £ then he may be swayed... do you have any additional information on Xen and DRBD? (I'm reasonably familiar with CentOS... what version did you use?)

 

Also - why did you not use Server 2003 R2 and mirror using DFSR and Virtual Server? Any particular reason?

 

Az

[dhicks]

do you have any additional information on Xen and DRBD? (I'm reasonably familiar with CentOS... what version did you use?)

 

CentOS 5.1 (I see 5.2 is now out). I tried Ubuntu 8.04 but found it kept crashing after the Xen kernel had been installed. Installing the Xen kernel on CentOS was simply a case of picking "virtualisation" from the install menu. I had a little bit of trouble installing DRBD on CentOS - you install DRBD via Yum and (if I remember correctly) it replaces the Xen-enabled kernel or something. I wound up installing DRBD on a separate CentOS system so I could get the required DRBD kernel module, then simply installing that kernel module on the CentOS machine with Xen running on it.

 

The DRBD manual is all you'll ever need, it is very clearly written, with examples and everything. I bought "Running Xen" after reading the review on Slashdot, but wasn't all that impressed as by the time I'd actually installed Xen on a couple of machines I figured I knew everything it explains, it doesn't get to more advanced stuff (like, how do I move a Windows 98 machine to run on Xen?).

 

I have a bunch of very rough notes which I'll try and tidy up and get on the Wiki at some point.

 

why did you not use Server 2003 R2 and mirror using DFSR and Virtual Server?

 

It costs money. You have to have servers with Windows licenses, which we don't have. Also, I like the way Linux organises its "disk protocol stack" - i.e. as a set of interchangeable protocols that you can kind of slot together as you wish. If I understand this right, DFSR replicates folders. DRBD replicates entire file systems, regardless of file system. It acts as a Linux block device, interchangeable with other block devices. I'm thinking of adding an encryption layer on top of DRBD, which will (hopefully) be a case of simply telling Linux to create another block device from the DRBD device and encrypt it.

 

--

David Hicks

Edited July 2, 2008 by dhicks

[azrael78]

Ah okay - thanks for that info though.

 

We have Windows licensing all over the place here, so we'd probably end up using DFSR and Virtual Server - if we use virtualisation at all that is.

 

Az