Veeam - I'm going to scream!

[eddyc]

Morning all,

 

I have spent the past three months on and off building our image for the summer on Veeam. The capture of the image failed, so I thought fine. I'll just roll back the snap shot and try again. Oh no! You can't do that as the restore fails. I've actually tried restoring other backups offsite (DC and mail server) and they worked without an issue. I don't really want to have to rebuild the image myself as I have spend ages building it. If anyone has any bright ideas I would really appreciate it. A copy of the logs are below and I do have a ticket open with Veeam support (which has now been opened for a week with little help!).

 

Thanks in anticipation, eddyc

 

 

25/07/2013 09:51:54          Starting restore job

25/07/2013 09:51:55          Locking required backup files

25/07/2013 09:52:07          Queued for processing at 25/07/2013 09:52:07

25/07/2013 09:52:10          Preparing next VM for processing

25/07/2013 09:52:09          Required backup infrastructure resources have been assigned

25/07/2013 09:52:10          Using target proxy VMware Backup Proxy [nbd]

25/07/2013 09:52:21          Starting restore agents on server "esxhost1.local"

25/07/2013 09:52:21          5 files to restore (100.0 GB)

25/07/2013 09:52:55          Restoring '[MACHINE_Store] MASTERIMAGE/MASTERIMAGE.vmx'

25/07/2013 09:53:16          Restoring file 'MASTERIMAGE.vmxf' (0.3 KB)

25/07/2013 09:53:41          Restoring file 'MASTERIMAGE.nvram' (8.5 KB)

25/07/2013 09:54:33          Registering restored VM

25/07/2013 09:55:17 Fail     Restoring disk '[MACHINE_Store] MASTERIMAGE/MASTERIMAGE.vmdk' (100.0 GB)

25/07/2013 09:55:29 Fail     Restore job failed Error: Client error: Zlib decompression error: [-3].

                             Cannot process [restoreDiskContent] command. Target descriptor is [vddk://

[sister_annex]

Which version of B&R are you using, virtual server backing up etc? Type of backup?

 

Is that all the logs say?

 

A quick google points at a corruption in the backup especially when you get a decompression error...

[seawolf]

I'm not exactly sure what you mean by "...building our image for the summer on Veeam." since Veeam is for backups. But, this error message smells of a corrupt backup to me. It could possibly also be a problem with the backup repository or the configuration of the backup infrastructure in Veeam. Have you checked the config or tried moving the backup files to another repository (another server or NAS)? Edited July 25, 2013 by seawolf

[sister_annex]

I presume he means he's been building a new server and using veeam to keep snapshot style rollbacks

[eddyc]

Which version of B&R are you using, virtual server backing up etc? Type of backup?

 

Is that all the logs say?

 

A quick google points at a corruption in the backup especially when you get a decompression error...

 

We are using Veeam Backup and Restore v6.5 on patch 3. This is running on Server 2012 (Physical Server)

[eddyc]

I'm not exactly sure what you mean by "...building our image for the summer on Veeam." since Veeam is for backups. But, this error message smells of a corrupt backup to me. It could possibly also be a problem with the backup repository or the configuration of the backup infrastructure in Veeam. Have you checked the config or tried moving the backup files to another repository (another server or NAS)?

 

Sorry that should have said on VMWare on our virtual environment. We build up the image, snapshot it, run sysprep and capture and then roll the image back so that the re-arm count is never reached. I have moved the backup files to another backup server and imported them. They verify fine but then still fail to restore.

 

Veeam support suggested an instant VM recovery (which runs it from the backup repository) and then migrating the VM live over to the production environment. The live instant recovery machine is fine but then the HOT migration back onto the virtual environment fails.

[eddyc]

I presume he means he's been building a new server and using veeam to keep snapshot style rollbacks

 

That's the one! I'm having a bad day but I'm glad you knew what I meant! :-)

[seawolf]

We are using Veeam Backup and Restore v6.5 on patch 3. This is running on Server 2012 (Physical Server)

 

Right there is most likely your problem. There have been many issues reported with Veeam and Server 2012, I would suggest you try moving your backup repository to another server (2K8R2 maybe or a NAS). The problems seem to be with 2012 and SMB/CIFS.

[eddyc]

Right there is most likely your problem. There have been many issues reported with Veeam and Server 2012, I would suggest you try moving your backup repository to another server (2K8R2 maybe or a NAS). The problems seem to be with 2012 and SMB/CIFS.

 

I did build a temp server 2008 r2 box with veeam and moved everything over to that with the same result too :-(

[seawolf]

BTW - I use a 2K8R2 server for Veeam and a FreeNAS server for the repository. It works brilliantly and with FreeNAS you can replicate everything to another FreeNAS server (offsite) for no extra cost (licenses).

[seawolf]

I did build a temp server 2008 r2 box with veeam and moved everything over to that with the same result too :-(

 

I think there is a 90% chance something's borked with your backup then. Hopefully not, but the consistency of the error is not looking positive. Good luck with it, hopefully Veeam support can help.

[eddyc]

Veeam support emailed me back to say that they would ring me at 3PM to remote in to me.

 

What excellent service! They have restored the backup to the last snapshot without an issue. There is a known issue with v6.5 on Server 2012 - so they replaced a couple of the EXE files in the installation with a different version which has resolved the issue.

 

:-)

[sister_annex]

Good to hear, I have to say Veeam support is nothing short of excellent - all the times i've had to contact them they have been prompt and very knowledgeable