Backup/Restore Hyper-V Domain Controller

[Koldov]

Is there a 'free' backup solution that is both VM and DC aware...?

 

Would this do it and if so has anyone actually restored a VM DC using it?

 

https://www.veeam.com/virtual-machin...tion-free.html

 

Had a quick look on the Veeam forum and found a similar question (specifically mentioning "it's a Domain Controller so AD must be in-tact", here were the first few answers:

 

"You will be able to backup with free edition, but there will be no central management."

 

"You can use Veeam Backup & Replication Community Edition 9.5 to backup your Hyper-V vms. Cheers!"

 

"With Veeam Backup & Replication Community Edition you can backup up to 10 VMs for free."

 

[Davit2005]

I have used Veeam to backup DCs and restored AD objects. If you have more than one DC I would personally not go down the route of restoring I'd look at building a new DC to replace the broken DC and if that DC had the schema roles etc. you can seize the roles (did that before many years back). If you have only one DC it should be fine for the job. I'd always be keen to have more than one DC (both Global Catalog and not RODCs)where possible for anything but very small environments on different hypervisor hosts. Edited July 14, 2022 by Davit2005

[Koldov]

So there's no 'real' point in taking a back up/snapshot/checkpoint of these Hyper-V DCs...?

 

They are just throw away instances and if one goes badly wrong just spin up another one?

[jthompson]

When I last used the free edition of Veeam B&R, it just let you do zip backups of VMs, with no option to schedule automatic backups or to do incremental backups. There were no application-aware restores of AD either (i.e. restore individual AD objects from a backup), being limited to restoring the entire VM. That may be enough to fit your needs, though.

[jthompson]

So there's no 'real' point in taking a back up/snapshot/checkpoint of these Hyper-V DCs...?

 

They are just throw away instances and if one goes badly wrong just spin up another one?

 

If your VMs get halted ungracefully and end up with corruptions, restoring them from a recent Veeam backup is preferable to having to rebuild them. That could be if the host crashes, or fails to suspend or shut the guests down properly when it's going for a reboot. Rebuilding DCs is a pain in comparison.

[zedster]

I'm using the latest version of VEEAM community and am able to backup VM's on a schedule and restore accordingly. However, I've also been pointed at this - which I want to stress I've not used - but looks like it may work if you have a suitable QNAP NAS - Hyper Data Protector | Backup unlimited VMware® and Hyper-V environments for free | QNAP (UK)

[Davit2005]

Defo No I'd still want a backup. If your AD environment goes badly wrong you have that option. Jus personal choice for me would be to build a new DC if one DC turned bad as a first choice after checking the existing AD for errors of course.

 

This is an older document but it goes though a bit about restoring DCs with Veeam

 

https://www.veeam.com/blog/how-to-recover-a-domain-controller-best-practices-for-ad-protection.html

 

License permitting I find it is hard to beat Veeam as a product for functionality, I may be biased as I used it for 4 years and upgraded all the way in steps from version 6 to 9.5. I no longer work in systems/server/AD though so up to date working knowledge of Veeam is missing. I did use it to restore AD objects and groups very successfully, they have made that so easy since version 6 where you had to spin up a virtual lab and then boot the backup into this isolated lab where it could write back to the DC. You could even spin up exchange in the same fashion and schedule testing of backups.

Edited July 14, 2022 by Davit2005

[Koldov]

Obviously I don't have the knowledge to explain properly, but I thought it would be a reasonably simple thing to do.

 

Backing up a Hyper-V isn't the main issue (any modern back-up software should be able to do that), but the DC part especially AD is very different and apparently needs specific things to happen in the back-up to make AD aware it is being backed-up... once it is restored it should then know it has to get updated from another authoritative DC and not start until it has.

 

I have become slightly wary as recently I did what I thought was a minor alteration to change from dynamic to fixed disk. Shut down the VM properly and then converted it to fixed. This unexpectedly created a new disk for some reason, so I attached the VM to it and booted up... OK so far... Shrunk the partition, then shut down the DC VM properly (again) and shrunk the disk. It all came up OK or so I thought but a little later it started getting refused replication and it seemed to point to a USN rollback, the only advice I found to clear it was demote and promote (which was a pain but as it was only a secondary DC and only a VM it wasn't that bad - primary was still on the host).

 

I don't really understand why it happened as the information on the disk should have all been the same as before it was converted and shrunk, nothing changed inside the VM, in the OS, or the AD part, only really the Hyper-V manager knew what happened...

 

Anyway as I said it made me think (that as a lot of the research I did seemed to suggest this is a fairly common scenario if you restore VM DCs and the backup wasn't VM and AD aware) that I would need to do a lot more research on how to do it.

[mavhc]

https://helpcenter.veeam.com/docs/backup/vsphere/tools_quiescence.html?ver=110

 

quiescence is the key word

 

https://dirteam.com/sander/2021/05/03/knowledgebase-vmware-tools-quiescence-corrupts-active-directory-backups/

 

https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/get-started/virtual-dc/virtualized-domain-controller-architecture

[round2it]

altaro vm backum might work for you they have a free verson

 

but do not skimp on backups its not worth it

 

the school should recognise this and ensure you have enough funds to effectivly have good backups as part of your business continuity plan

 

 

I use altaro here with a qnaps and wasbi offsite bucket.

 

after a failed cluster i managed to get everything back up and running within a day so defo money well spent its actually very cheap

 

 

Its an option different to veeam

[mavhc]

use a filesystem with snapshots built in and then sync those snapshots to a remote server.

 

I snapshot every virtual drive every hour and sync them to a remote machine, can delete old incremental backups no problem, can mount any backup date, and it's set so you can only remotely add snapshots, never delete them

[Koldov]

Honestly, the more I try to research it the more complicated it gets...

 

This from MS (obviously I've paraphrased):

 

Supported methods to back up Active Directory on domain controllers...

 

Use an Active Directory-aware backup and restoration utility that uses Microsoft-provided and Microsoft-tested APIs.

 

Use an Active Directory-aware backup and restoration utility that uses Microsoft Volume Shadow Copy Service APIs

 

Restore the system state.

 

Software and methodologies that cause USN rollbacks...

 

Starting an Active Directory domain controller whose Active Directory database file was restored (copied) into place by using an imaging program

 

Starting a previously saved virtual hard disk image of a domain controller. The following scenario can cause a USN rollback:

 

1. Promote a domain controller in a virtual hosting environment.

2. Create a snapshot or alternative version of the virtual hosting environment.

3. Let the domain controller continue to inbound replicate and to outbound replicate.

4. Start the domain controller image file that you created in step 2.

 

There's a lot more, but basically reading this it seems that there's a lot that could go wrong with any kind of restoration of a snapshot/image...

 

Yet, in the MS link @mavhc posted it seems that a Hyper-V DC has its own check on boot-up and can't really go wrong (with Generation-Id and virtualisation safeguards)... as long as the DC knows it is not in sync with the latest changes (so it can't be a straight-up clone or a back-up that didn't trigger the DC to let it know it was being backed-up).

 

It seems from the other link Veeam is the way to go, but I wonder even if I do a back-up and test restore it to another machine (off-Domain) how do I check it will behave in the way it should without actually letting it back on the Domain...?

 

Typical behavior that occurs when you restore an Active Directory-aware system state backup...

 

When Active Directory is restored on a domain controller by using the APIs and methods that Microsoft has designed and tested, the invocation ID is correctly reset on the restored domain controller. domain controllers in the forest receive notification of the invocation reset. Therefore, they adjust their high watermark values accordingly.

 

Also, are Hyper-V snapshots/checkpoints AD/DC aware?

Edited July 15, 2022 by Koldov

[Davit2005]

Try not to over think it.

 

Evaluate the risks to AD, set backup retention (don't skimp on backups) and regularly monitor AD for issues. Before doing any AD work i.e. bringing in new domain controllers, check AD for any existing errors.

 

Work/test through some DR scenarios.

[mavhc]

They're pretty much all aware of quiessence, they call VSS to make the snapshot, that tells AD/MSSQL/anything else to finish their transactions and write to disk

 

The AD failsafe on restore is a newer thing, so check specifically for that.

 

But if you have a VM that only does AD, don't bother restoring it, just script installing a new one

[Koldov]

Yeah, that does seem to be the general consensus...

 

So why bother with backing it up, if I'm never going to restore it?

 

Although I do need to make sure I have just the FSMO DC in a backup for complete DR, but in that case restore it first and it will be the only one to start with, so it doesn't matter if it's 100% accurate to the others (because there won't be any) and any I create after that will be blank and replicate from it.

Edited July 15, 2022 by Koldov

[Davit2005]

You still need a backup for DR. What happens if you lose your hosts totally or something else happens. Consider you lose both DCs.

 

The implications of Cloud email systems connected to AD etc. Other data that may be stored on the DC such as home areas etc. Other services i.e. DNS, DHCP commonly installed on DCs.

 

Personally I'm a believer in belts and braces when it comes to systems the more options you have the better.

Edited July 15, 2022 by Davit2005

[mavhc]

Yeah, just need to backup 1 of them, not all of them

[Koldov]

And therefore don't need to worry so much about consistency, if it's just for a total DR job.

[ITGURU]

Should always have at least 1 x physical domain controller so you have a domain/can restore a domain before other systems including backup severs if they are domain joined!

[Primus]

Should always have at least 1 x physical domain controller so you have a domain/can restore a domain before other systems including backup severs if they are domain joined!

 

It's a matter of personal preference. You do not need to have a physical one now but some might prefer to have one.

[mavhc]

Really what you should have is a backup system that you've tested you can disaster recover from, without a load of dependencies.

 

If we're talking on site backup, having a domain joined backup system sounds like an excellent way to get your backups encrypted by the russians, if you can remotely log into your backup system that's a bad idea, it should have 1 network service available, that only accepts backups, anything else should only be possibly via physical access behind multiple locks and passwords

[houlta]

You can use UrBackup - Client/Server Open Source Network Backup for Windows and Linux which is free open source and run it on a standalone non domain joined box and have the agent on the machine you want to backup. Job done.

It even runs on linux if you want.

 

Then just use a rented license of veeam via a cloud immutable backup, it's much cheaper.

That way you have on-site and you have cloud and it costs the least as you're not paying veeam for a full license.