Server Failure

[Chuckster]

Let's say hypothetically that one of the DCs with all the FSMO roles was to go up in smoke.

 

How would you go about setting up a new DC with your AD, DNS, DHCP etc all intact?

 

Am I right in saying that you would need to install Win 2003 on a new server and from your backup tapes, restore the System State and every other local drives from it. Reboot the server and bob's your uncle?

 

Is that all there is to it and am I being naive?

[FN-GM]

Nope .

 

I would seize the roles to the working DC as shown here - Using Ntdsutil.exe to transfer or seize FSMO roles to a domain controller that will give your working DC all your roles.

 

I would then install Server 2003 on the new server then remote it to a DC when you have done that. Its very easy to do i have done it a few times.

 

Restoring the backup can cause problems in AD. I would only restore backup of AD if all DC's are done for what ever reason.

[EduTech]

AD, you can just have a secondary domain controller so that if your main DC falls over users can still autenticate agaist that server.

 

DNS and DHCP - not really my strong point (we have no failover for it yet in place)

 

but i know some people have configured DHCP and DNS on the secondary DC but there disabled, when the main dc goes down enable both DNS and DHCP on your secondary DC so clients can get a new IP etc. (dont quote me on that though)

 

James.

[Chuckster]

Nope .

 

I would seize the roles to the working DC as shown here - Using Ntdsutil.exe to transfer or seize FSMO roles to a domain controller that will give your working DC all your roles.

 

I would then install Server 2003 on the new server then remote it to a DC when you have done that. Its very easy to do i have done it a few times.

 

Restoring the backup can cause problems in AD. I would only restore backup of AD if all DC's are done for what ever reason.

 

What if the actual server has burnt down.. would I still need to use the Ntdsutil.exe?

 

 

 

 

AD, you can just have a secondary domain controller so that if your main DC falls over users can still autenticate agaist that server.

 

DNS and DHCP - not really my strong point (we have no failover for it yet in place)

 

but i know some people have configured DHCP and DNS on the secondary DC but there disabled, when the main dc goes down enable both DNS and DHCP on your secondary DC so clients can get a new IP etc. (dont quote me on that though)

 

James.

 

 

I have another DC that doesn't do DNS or DHCP. I need to know how to do this so that if the worse case scenario was to happen, I can enable the DHCP and DNS on the second DC server.

[FN-GM]

AD, you can just have a secondary domain controller so that if your main DC falls over users can still autenticate agaist that server.

 

DNS and DHCP - not really my strong point (we have no failover for it yet in place)

 

but i know some people have configured DHCP and DNS on the secondary DC but there disabled, when the main dc goes down enable both DNS and DHCP on your secondary DC so clients can get a new IP etc. (dont quote me on that though)

 

James.

 

Ideally you would have DNS running on both servers nd have your clients point at both servers.

 

What if the actual server has burnt down.. would I still need to use the Ntdsutil.exe?

 

 

 

 

 

 

 

I have another DC that doesn't do DNS or DHCP. I need to know how to do this so that if the worse case scenario was to happen, I can enable the DHCP and DNS on the second DC server.

 

You will need to install DNS and DHCP on the second server. As said up there DNS should be already on ideally

 

Restore a backup if - All your DC's are dead

 

use Ntdsutil.exe if - You have one good DC

[m25man]

You can ONLY restore your AD if your backup includes a valid systemstate and all of your system drive files.

 

You must regularly test restoring the system state to an alternate location.

I have on several occassions tried to restore the ntds.dit and systemstates only to be told that there are inconsistencies in the backup and it will not restore!

 

There are know issues with VSS and the Veritas volume shadow copy services that have made so called backups worthless when really needed.

 

NTDS Util and metadata cleanups are not for the faint hearted.

 

If you want a bullet proof way of protecting your AD have a simple PC set up as a DC somewhere and Ghost it regularly.

That way if the worse does happen you simply seize the FSMO roles and rebuild your domain from that image.

Forget thousands of pounds worth of DR software and devices.

At most £150 for a PC, £80pa for your server license and a copy of Ghost you are sorted.

One of these and a good backup of your other systems will get you back from almost anything.

[Zorba]

An idea I have been toying with but not tried yet was to run a DC in VM ware. Fire it up allow AD to replicate, shut it down and keep this on a usb hard drive off site, perhaps doing this once a week. Thoughts anyone? Would this work?

[irsprint84]

My opinion (this is for new servers coming onto the domain), install widows server, pacth it to the latest stuff at the time but dont promote it yet but clone it. If the server dies, build from the clone, install the few updates then DCPROMO it, cuts the time between cold install.

 

Its good people are looking at worst case senarios but these are rare, just check your backups regularly.

[Zorba]

My opinion (this is for new servers coming onto the domain), install widows server, pacth it to the latest stuff at the time but dont promote it yet but clone it. If the server dies, build from the clone, install the few updates then DCPROMO it, cuts the time between cold install.

 

Its good people are looking at worst case senarios but these are rare, just check your backups regularly.

 

Your right they are rare but what about when your school suffers a fire. Happened to a school just down the road. It ocurred during the school day and on the very day that the backup bag had been brought in to change the tapes (left in server room)!!

 

Luckily Fire Brigade brought fire under control before it reached server room and the servers suffered only some water ingress. They were left for a week with the covers off to dry out.

 

I try not to think about such a thing happening at our school but when I do I get a sick feeling in the pit of my stomach.

[irsprint84]

Your right they are rare but what about when your school suffers a fire. Happened to a school just down the road. It ocurred during the school day and on the very day that the backup bag had been brought in to change the tapes (left in server room)!!

 

Luckily Fire Brigade brought fire under control before it reached server room and the servers suffered only some water ingress. They were left for a week with the covers off to dry out.

 

I try not to think about such a thing happening at our school but when I do I get a sick feeling in the pit of my stomach.

 

I agree totally, never sit back, I m always reading up regularly how to deal with such circumstances.

[m25man]

An idea I have been toying with but not tried yet was to run a DC in VM ware. Fire it up allow AD to replicate, shut it down and keep this on a usb hard drive off site, perhaps doing this once a week. Thoughts anyone? Would this work?

 

Probably not the best idea as the replication partners will notice that it has gone off-line and will start to complain .

Probably better to keep the VM DC running at all times just snapshot it and take the archive off site.

 

Use Hyper-V here for our own stuff,

 

1x HP ML350 G5 2 x QC 3Ghz with 32GB Ram running 2k8 DataCentre and Hyper-V

2 x 4 Port Server NIC's 6 x 500GB Enterprise SATA

iSCSI external Array 2.5" USB Backup devices.

1 x DL380 G5 2 x QC with 32GB & 8x 146 SAS 1 x 4 Port Nic

 

SBS2008 Exchange 2007 & 2010 (1x Production & 1x Beta)

EBS 2008 (3 Server Rig)

MS Home Server, Various Linux and Windows rigs to play with.

Virtualisation.... it's the only way to play!

[irsprint84]

Probably not the best idea as the replication partners will notice that it has gone off-line and will start to complain .

Probably better to keep the VM DC running at all times just snapshot it and take the archive off site.

 

Use Hyper-V here for our own stuff,

 

1x HP ML350 G5 2 x QC 3Ghz with 32GB Ram running 2k8 DataCentre and Hyper-V

2 x 4 Port Server NIC's 6 x 500GB Enterprise SATA

iSCSI external Array 2.5" USB Backup devices.

1 x DL380 G5 2 x QC with 32GB & 8x 146 SAS 1 x 4 Port Nic

 

SBS2008 Exchange 2007 & 2010 (1x Production & 1x Beta)

EBS 2008 (3 Server Rig)

MS Home Server, Various Linux and Windows rigs to play with.

Virtualisation.... it's the only way to play!

 

Alot of people say about snapshotting the VM's, if there was a DC disaster they would put the snapshot back, though most people say this causes replication issues, would you do this?

[m25man]

Alot of people say about snapshotting the VM's, if there was a DC disaster they would put the snapshot back, though most people say this causes replication issues, would you do this?

 

You can use the snapshot ONLY if all other DC's are unavailable.

 

There are two types of AD recovery Authorative and Non Authorative.

Replication Partners keep track of transactions via USN's each partner knowing the status of it's neighbours.

 

What I was suggesting was that a snapshot of a valid DC could be used to construct a replica domain/forest in the event of a catastrophic failure.

 

We do it all the time in SBS Migrations, you see SBS only allows one SBS Server in a domain but allows multiple DC's

As everything inc Exchange SQL and ISA lives on an SBS box it's not easy replacing the hardware if it fails!

 

This unathorised method is known as the Swing Migration.

It is not supported by MS as "Cloning" the Domain SID is not exactly an endorsed practise.

However if you Google for SBS Swing Migration you will see it is very widely used by us die hard SBS Pro's....

 

In SBS we add a second DC and allow replication, then remove it by unplugging it.

Use NTDSUtil to remove the temporary DC from the production environment.

 

On the now detached DC we seize the FSMO roles this creates an exact replica of the AD Forest in the source Domain.

Then use NTDSutil to remove the origional DC references of the source domain.

Edit: Left out the important bit.....

Now using our working VM with the cloned AD we attach the new target hardware, designate this as another DC with the SAME NAME as the DC in the production domain... the most important bit.

 

Gracefully demote the VM.

 

Hey Presto, This Domain is now a perfect Replica of the source.

 

All of your user and computer accounts exist in both.

 

By using VM's in this manner you can prepare your new 2008 School Domain without ANY impact on your existing production domain.

 

When your happy with the off line setup just un plug your old servers and connect your new ones.

Nobody is any the wiser!

 

So now I have shared a carefully guarded secret and told you all how to deliver a zero impact Server 2008 Domain Migration I expect to see some serious Rep posted....

Edited May 3, 2009 by m25man

[Eric]

So now I have shared a carefully guarded secret and told you all how to deliver a zero impact Server 2008 Domain Migration I expect to see some serious Rep posted....

 

Geoff,

 

I hope that you'll pick up on this reply to a two-years-old thread ...

 

I've think that I've always understood the part of the swing migration process which you describe, but Jeff Middleton (of SBSmigration.com) also says that he can repeat the migration time and time again in practice, including file migration, until he is satisfied it all goes right, then do the migration for real (and still have the capability to roll-back.) Please can you shed some light on how this part is done?

[edutech4schools]

Another thing you can do, although I will get shot down by others on this site is to run vmware converter on the server and get a copy of your live machine and just leave this file somewhere safe. Then when needed simply install vmware server or player onto any pc and point it at the file to have you server backup and running in a virtual machine while you fix the real server.

 

I do this all the time, and have tested.

[zag]

Another thing you can do, although I will get shot down by others on this site is to run vmware converter on the server and get a copy of your live machine and just leave this file somewhere safe. Then when needed simply install vmware server or player onto any pc and point it at the file to have you server backup and running in a virtual machine while you fix the real server.

 

I do this all the time, and have tested.

 

Yep thats how I am backing up my main DC with hyper-v creating a vhd2disk thingy.

 

I do have another DC in my office though so would normally just sieze the roles from that one and install DHCP again.

[Chuckster]

Effectively the same route as I currently take but slightly different. I use ESXi and a script to backup the VM images onto a NAS drive. From there I copy them onto an external HDD. I have booted the images in VMware Workstation and it loads up fine. Both my DCs, AV & Print, SIMS, WDS/MDT and terminal server are VMs. File servers where user home drives are kept and shared drives are sitting on a physical box that simply back up to tape.

 

I did give Hyper-V a try but I didn't like it because it's like VMware Server or Workstation. My preference is to have a simple Hypervisor like ESXi so there's less room for it to go wrong. *touch wood*

[edutech4schools]

And to backup or move DHCP I do this:

 

export = netsh dhcp server export c:\dhcp.txt all

import = netsh dhcp server import c:\dhcp.txt all