ESXi Backups

[Hightower]

We are using the ghettoVCB script that is freely available to backup the VMs on our ESXi server. We backup to NFS (FreeNAS) which is then picked up on a night by our tape drive. Our VMs are setup with thin provisioning, and the backups are taken in a thin format (as option is available in ghettoVCB script). When browsing the datastore using vSphere the size of the backups displays as the thin size (i.e. 2GB instead of the provisioned size of 50GB), but when browsing to the SMB or backing up using BackupExec they both see the files as their provisioned size (i.e. 50GB instead of 2GB that is actually being used).

 

This means that our VM backup is around 200GB instead of a more suitable 20/30GB. Why are these files showing as a different size to anything but the vSphere client, and how to I go about changing it so that others (i.e. BackupExec) see it for its true size?

 

Thanks,

[Ashm]

Same thing here, we have an ubuntu box and backup using ghettoVCB over nfs to that. Browse through SMB and the size of the VM is the full provisioned size, where as checking the size on the server itself displays the actual used space which is much lower.

 

The easiest way i've found to reduce the backup size is to compress the backups using gunzip/7zip etc. This will then create a compressed archive that is at least as small as the thin size (or actual used space).

[Hightower]

Same thing here, we have an ubuntu box and backup using ghettoVCB over nfs to that. Browse through SMB and the size of the VM is the full provisioned size, where as checking the size on the server itself displays the actual used space which is much lower.

 

The easiest way i've found to reduce the backup size is to compress the backups using gunzip/7zip etc. This will then create a compressed archive that is at least as small as the thin size (or actual used space).

 

Can the ESXi server carry out ZIP tasks? I wouldn't have thought the required software would be installed. Just because the easiest way to ZIP them up would be to add a ZIP command to the ghettoVCB script.

[Ashm]

You can get the ESXi server to compress the backups with gzip+tar. There is an option in GhettoVCB with experimental support. See the ghettoVCB page for more info - VMware Communities: ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5 and 4.0+

 

The way I do it is to run ghettoVCB remotely through a script from the Ubuntu box, the script then zips up the ghettoVCB backup using 7zip (so it's easy to restore from a Windows box). The way I see it, this way the ESXi server is under less load as it only has to copy the VMs to be backed up as opposed to compressing them as well.

[Duke]

Sorry to hijack this thread a little, but I'm looking at setting up our ESXi backups currently. When we fully virtualise it'll be on ESX and I'll get the Backup Exec agent to handle the backups, but for now I just need to get our development ESXi box properly backed up.

 

Are people generally happy with GhettoVCB? It looks simple enough but are there any major issues to be aware of? The VMs are running from a SAN over NFS, so I'll probably either just back them up to a different share on that SAN and get Backup Exec to grab them via CIFS from there to tape, or I'll connect another iSCSI SAN to the ESXi host and use that for backups. Anyone see any major issues with that?

 

Many thanks in advance!

 

Chris

[Hightower]

Sorry to hijack this thread a little, but I'm looking at setting up our ESXi backups currently. When we fully virtualise it'll be on ESX and I'll get the Backup Exec agent to handle the backups, but for now I just need to get our development ESXi box properly backed up.

 

Are people generally happy with GhettoVCB? It looks simple enough but are there any major issues to be aware of? The VMs are running from a SAN over NFS, so I'll probably either just back them up to a different share on that SAN and get Backup Exec to grab them via CIFS from there to tape, or I'll connect another iSCSI SAN to the ESXi host and use that for backups. Anyone see any major issues with that?

 

Many thanks in advance!

 

Chris

 

I edited the ghettoVCB so at the end it set the permissions to 755 (so BackupExec can swoop in and copy them off). Other than thin backups showing as their provisioned size (which I'm looking into) to BackupExec, the script works perfectly

[Duke]

I edited the ghettoVCB so at the end it set the permissions to 755 (so BackupExec can swoop in and copy them off). Other than thin backups showing as their provisioned size (which I'm looking into) to BackupExec, the script works perfectly

 

Cool, sounds good! Thanks, much appreciated.

[nicholab]

VMware Backup, ESX backup & replication, VMware Restore with Veeam Backup

 

Veeam is used by a top London school I went to see.

[Duke]

VMware Backup, ESX backup & replication, VMware Restore with Veeam Backup

 

Veeam is used by a top London school I went to see.

 

Yeah, I've heard good things about them. Unfortunately I haven't got any budget for this so need a free solution. When we go beyond just development I'll be able to afford to do it properly, so that may be a better option than Backup Exec (which isn't cheap anyway).

[Duke]

Well, I think ghettoVCB is finally going to work, tonight will be the first properly scheduled backup so we'll see.

 

A few things for other people trying this:

 

1) No matter what I used to download the scripts onto ESXi, they came up with DOS formatting and lots of ^M's. If you don't want to use sed or vi to remove these, dos2unix on another Linux server is your friend, then use SCP to move them to ESXi.

 

2) The ESXi root filesystem isn't persistent between reboots. This isn't clearly mentioned in the ghettoVCB docs (although crontab resetting is) and the examples show the script being stored in the root user's home folder. This won't work and you need to store all your data in /vmfs/volumes/somewhere.

 

3) The examples for putting ghettoVCB.sh into crontab seem to show an old formatting of

 

4) /var/spool/cron/crontabs/root is read-only for root, you need to chmod 644 it before making changes.

 

5) For me at least, on 4.0 U1, running /bin/busybox crond just gives me another instance of busybox. I actually had to run crond itself which launches as busybox anyway. Run cat /var/run/crond.pid if you're having issues to check it's working.

 

Probably just me having issues, but hopefully this'll help someone.

 

Chris

[Hightower]

1) No matter what I used to download the scripts onto ESXi, they came up with DOS formatting and lots of ^M's. If you don't want to use sed or vi to remove these, dos2unix on another Linux server is your friend, then use SCP to move them to ESXi.

 

I used my Moodle box for this (Ubuntu) as I couldn't figure out how to do it in ESXi. Once I removed the ^M's it was already on web hosting for me to re-download to the ESXi box.

 

2) The ESXi root filesystem isn't persistent between reboots. This isn't clearly mentioned in the ghettoVCB docs (although crontab resetting is) and the examples show the script being stored in the root user's home folder. This won't work and you need to store all your data in /vmfs/volumes/somewhere.

 

I put mine in the sbin directory as far as I can remember.

 

4) /var/spool/cron/crontabs/root is read-only for root, you need to chmod 644 it before making changes.

 

Remember to change the permissions back after you have updated the cron jobs.

 

I don't think it was just you with these issues though as I experienced three of them when I installed - so good tips that will hopefully help someone

[Duke]

re 2) - Yeah, that probably would have worked for me too. I made the mistake of making a /ghettovcb folder in the root for all my scripts and settings, then did a reboot at the end and it wiped it all! I think it's somewhere between a non-persistent set of areas on the file system and some places that get reset to the defaults.

 

re 4) - Good point, although it will reset them after a reboot I believe. It re-creates the default file when it boots up with the default permissions then rc.local makes your changes. Worth checking though.

 

Fun and games, can't complain for free though. Next week I get to try the restore script... Good to have the data on two separate SANs though.

 

Thanks for the help!

Chris

[mahmoud]

Hello

hw r u my friends iam new member here in EduGeek

thanks for this subject but i want to ask about anyone have serial or keygen for veeam backup and replication, cause i want to try it before buy it

[Duke]

I'm pretty sure requesting a serial or keygen would be against forum rules (and illegal)...

 

I'd suggest you speak to your nearest reseller or Veeam who can probably arrange a trial for you.

[Sandman]

esXpress is a good ESX backup solution.

 

ESXi support is currently under development so keep you eye out for that too!

[diggory]

Sorry to hijack this thread a little, but I'm looking at setting up our ESXi backups currently. When we fully virtualise it'll be on ESX and I'll get the Backup Exec agent to handle the backups, but for now I just need to get our development ESXi box properly backed up.

 

Are people generally happy with GhettoVCB? It looks simple enough but are there any major issues to be aware of? The VMs are running from a SAN over NFS, so I'll probably either just back them up to a different share on that SAN and get Backup Exec to grab them via CIFS from there to tape, or I'll connect another iSCSI SAN to the ESXi host and use that for backups. Anyone see any major issues with that?

 

Many thanks in advance!

 

Chris

 

I've found that if you change the number of backups you want kept before it rolls over the first one, it doesn't keep track of the numbers properly, so you may need to manually delete a few backups if you change the numbers.

 

The gzip option works, but takes a while.

 

If you leave a VM with a snapshot on it (naughty) it won't get backed up.

 

It does work on ESX even in a cluster (although you have to run it on each machine in the cluster - as it will only backup vms running on the box it's running on).

 

Works well for us though, it's free and I've recovred vms with it without a problem

 

PS. running our as a cron job and backing up to a NFS server.

[diggory]

Well, I think ghettoVCB is finally going to work, tonight will be the first properly scheduled backup so we'll see.

 

 

2) The ESXi root filesystem isn't persistent between reboots. This isn't clearly mentioned in the ghettoVCB docs (although crontab resetting is) and the examples show the script being stored in the root user's home folder. This won't work and you need to store all your data in /vmfs/volumes/somewhere.

 

 

Chris

 

I put my script in the sam location as my backups - ie an actual vm store, rather than on the host itself.

I found most of the info I needed at a now dead page, but for ghettoVCB newbies there's more info on the ghettoVCB page:

VMware Communities: ghettoVCB.sh - Free alternative for backing up VM's for ESX(i) 3.5 and 4.0+ - Updated 04/20/2010

[diggory]

Using VMWare data recovery for our ESX now. It's free for ESX enterprise. So not much use if you are running ESXi (then ghetto is for you).

It does have de-duplication - which works very well and apparently supports CIFS (haven't tried that).

 

Chart of versions that licensed for data recovery: http://www.vmware.com/products/vsphere/buy/editions_comparison.html

 

Data recovery info: http://www.vmware.com/products/data-recovery/

Edited May 6, 2010 by diggory