I just wanted to let others know my recent experience with our school DC's and hyper-v, and hope this will help save some one the horror of almost losing AD.
I will keep it short, (parts and steps may have been missed)
We are running on 2 blade servers each one has a DC, replicating... The problem was when for a reason or other we needed to restore a backup. Now any one who uses VMWare, Hyper-v or indeed was at the last educonf will know you can take a snapshot of the server in its current state and then have the option to rollback to a previous state if you have a problem.
Such a time came, so with out hesitation a rollback was done. This is when the problems started.
Our first problem was we got an error stating netlogon was paused, and replication failed
After several min of looking at logs... we decided another rollback (Still unaware this was the problem) and still not fixed the netlogon issue.
We decided to isolate, demote and remove our backup DC then rebuild it. Having done so we rebooted both servers only to find on our DC had no users, workstations. Both of us (network manager n' me) in order then then
At this point we found a few posts from other forums (Yes, shock to me that others are out there) but Edugeek FTW) that having a V-DC and using snapshots, rollbacks... would make issues like the one we where having.
As we did not have a "Normal" backup of our DC another rollback was done on both to a time that where only a few min apart from each other, in the mind set well it cant get much worse then loosing ones AD.
Thankfully, this worked. we where still in the same spot of not having a working netlogon and replication but all the users... where back.
more digging and we found a simple command to fix the connection for replication.
repadmin /options -DISABLE_INBOUND_REPL
repadmin /options -DISABLE_OUTBOUND_REPL
both restored the connection and forcing the netlogon service to run, so users where back and they could log on
now we are still having issues with the server and NM is in the proses of working out if a rebuild of the DC or some other option is out there, but what i want to tell you is definitely go for some sort of V-DC just to read this **LINK** before you attempt it.
I hope i have helped some one buy sending out this warning, that or every one already knows the problem and it was an oversight on our part.
Be careful with snapshots for backups too. Certainly for vmware keeping a snapshot for anything other than a short time is a bad idea due to the massive redo disks that can be generated which can then take a very long time or fail to merge.
At minimum you want to do a system state backup of DCs even when using other imaging products for backup.
As you have found, you never want to snapshot DCs if there are multiple, with a single you can just about get away with it.
speckytecky (19th August 2009)
Thanks for the info, I knew about the general snapshot limitations with DC's but the info about export etc is v useful.
Thanks for this - while we don't have any V-DC's yet, it's only a case of time before management decides to cut back on our server farm and request we virtualise even more... great idea, nice technology - but on some things, virtualising adds a few nice things (like snapshotting), great for workstations but I'm all for taking a full backup of a server and not using snapshot technology.
However I will be sure to keep this article in mind when the day of V-DCs and someones' 'playing around with said DCs' happens...
Microsoft have always said that you should never image a domain controller as a backup means. In the past, I'd guess it wasn't easy to do this (would have had to boot into WinPE or whatever to run Ghost etc) but snapshotting makes it much easier so this is a timely reminder that you mustn't ever try and use a snapshot/image type backup.
I'd guess the only exception might be if you've had a total disaster and lost everything - bringing back one image could work. (It's also OK if you've only got one DC but no-one runs a real network like that :-))
Yes they have but not very loudly... this precise topic has cropped up a couple of times on here to my knowledge.. suspect searching for USN will find them, but it's *definitely* worth repeating as it's clearly not widely understood and more and more folk will be getting involved with VMs.Microsoft have always said
I reckon (YMMV) the safe way to do DC snapshots is to shut down all your DCs and then snapshot them - and if you do want to roll back and it's within the tombstone period, shut them down again and revert them all back to that set of snapshots, then of course start them up. Not sure how useful it is, but it's the kind of thing I might consider doing immediately before a round of serious upgrading.
The other issue with snapshots is of course performance - can't comment on how much it hurts but I've been assured by a serious expert that it does (and the hit obviously increases with the number of snapshots).
Couple of handy links here.
Deployment Considerations for Virtualized Domain Controllers
Virtualized Domain Controllers and Replication Issues
How to detect and recover from a USN rollback in Windows Server 2003
It's also worth remembering tombstone lifetime issues if restoring an old image or snapshot. As DMcCoy said a good system state backup is essential.
Last edited by cookie_monster; 18th August 2009 at 08:43 PM.
You can restore snapshots and system state backups but that means more downtime and also many more possibilities for issues with users who have changed their passwords or been added since the last backup. Worse if some of the stations have aumotaticly refreshed their machine passwords in the background during that time kicking them unglamourously off the network (modern Windows OSs do this with some regularity in the background to increase security).
AD itself although being a database is infact comparitivly light it is the the concern of consistancy and up to the minute data preservation that is a factor in many smaller environments.
I just would like to add that you should never take a snapshot then try increase the vhd. This breaks the snapshot.
Lucky for me I had a back up and could restore the VM.
I must of been working without having a cup of tea that day as I thought I had merged the snapshot.
I would say that no one would run a large network like that I'm certain there are many thousands (possibly millions) of Windows SBS single server setups out there.Originally Posted by srochford
It's also OK if you've only got one DC but no-one runs a real network like that :-))
I take it you're not an SBS fan then? (the server software I mean not the crazy boat people)
There are currently 1 users browsing this thread. (0 members and 1 guests)