Hyper-V Checkpoints Corrupted VM

[Zoom7000]

I spent the first few weeks building a brand new virtual server domain environment on a physical Windows Server 2016 server. Got everything done and dusted and all was working fine, until this morning. In another thread I posted yesterday, I talked about backup strategy and possibly looking at Veeam. This morning, I decided before I'd do anything, I'd take a checkpoint of the domain controller VM (DC01). Nothing happened. I tried again and nothing happened. I connected to the VM and the VM totally crashed. Couldn't ping the servers IP or anything. I tried to "Turn Off" the VM. Still nothing. So, I decided to reboot the physical PC. As soon as it came back up, I tried to boot up DC01 and got the following message:

 

"An error occurred while trying to start the selected virtual machine.

 

DC01 failed to change state."

 

It felt like the VM had corrupted, so I created another one and re-attached the VHD files from DC01 and all was working fine. So, I installed Veeam free edition on to the physical server and tried to take a VeeamZip backup. The first thing it tries to do is create a checkpoint. As soon as it did, it failed. I checked Hyper-V and sure enough, the VM had completely frozen again. It seems like checkpoints are killing my VMs and as it stands, I have no DC or any way to back it up.

 

Before I try to create the whole domain again from scratch in case it's the VHDs causing the issue, is there anything I can do to recover and is there any reason why this is happening?

[Zoom7000]

Just for reference for anyone stumbling across this in the future, managed to get to the bottom of why this was happening. Basically, it isn't a good idea to create checkpoints on a domain controller. Hence why the corruption was occuring. Also, I was using the wrong Veeam product. I had installed Veeam Backup and Replication, whereas the more suitable option was Veeam Endpoint Backup. This appears to work like a more traditional backup program which you install an agent on the computer that needs to be backed up and then Veeam creates an image of the entire computer which can then be restored to a new VM.

 

Presumably it would also work for a physical PC to perform a P2V. Not tested that yet though!

[NB9457]

Was about to say, checkpoints on DC VMs is bad, but you worked that out!

 

I manually back up one of My DCs around every 30 days, just shut it down, copy the VHD file to the backup system and the rest of the VMs and servers backups are automated.

[Zoom7000]

I manually back up one of My DCs around every 30 days, just shut it down, copy the VHD file to the backup system and the rest of the VMs and servers backups are automated.

Just out of interest, how do you automate your backup? What do you use?

[NB9457]

Just out of interest, how do you automate your backup? What do you use?

I use Veeam Free for all VMs and Iperius Free as a file level backup, both hosted on separate HP Microservers, which then copy to external 10TB external HDDs for weekly offsite backups, I will also be taking advantage of Cloudberry and Google Drive soon as a free cloud solution, but due to not having the best download speeds, this will be for absolute worst case and maybe as a bit of an archive too. We also have shadow copies enabled too for live data so the end user can recover their own files.

 

As we don't have a SAN and are literally only just going virtual, I'm also planning on using an old server with some large HDDs and more RAM for HyperV replication of core servers, such as file and SIMS.

[cpjitservices]

Checkpoints in Hyper-V is a bad idea anyway and should only be used for test & dev, dont rely on them entirely. Also, watch the AVHDX files as they end up growing and growing and growing.

 

https://blog.workinghardinit.work/2014/08/25/3-ways-to-deal-with-lingering-hyper-v-checkpoints-formerly-known-as-snapshots/