NAS Box for backup

[Admiral208]

Hi everyone.

 

We currently use tapes for our backup solution. I would like to move this to a nas box and then use the tapes to do a Weekly backup which I can put in the schools safe.

 

Our current daliy backup is getting close to 1TB so it would need to hold at least a weeks worth of data. Can someone please recommend me a good one. We use Backup Exec, dont know if that makes a difference.

 

Is there anything I should be aware of doing this?

 

 

Thanks

 

James

[Dos_Box]

A 1TB backup!? Are you amending these tapes or is it a fresh complete backup every night?

[Admiral208]

Sorry, I rounded up. Our current backup is about 650GB but i'm allowing for growth.

 

Its a complete backup every night.

[glennda]

Do you run full backups each night? As Incremental would be alot better! We currently backup about 2.4TB each Friday to a custom built 28TB ubuntu server. Then each week night we backup the incremental stuff for the main file servers (full Sims Database copy though). If you want i can post the spec of the server which i bought (although if you want to run windows you would need to change the raid card as i run software raid.)

 

Overally it cost me circa 4K

[Admiral208]

ok thanks.

 

I didnt set up the backups, it was my predecessor. I like the idea of incremental backups. Would this work?

 

Monday - Full backup to NAS drive.

Tues -> Friday - Incremental backup to NAS.

End of month - Full backup to tape to store in safe.

[dhicks]

Monday - Full backup to NAS drive.

Tues -> Friday - Incremental backup to NAS.

 

Is it just user files you'll be backing up, not VM images or anything like that? If so, why not use rsync to copy the files from your main file server every night to your backup server, a different backup folder every day, but with the backup server running some kind of deduplication system (either a deduplicating file system or a script that does file deduplication). That way you could probably hold more than a week's worth of week's worth of backups, depending on just how many files are changed per day.

[Guest Guest]

ms dpm 2010 is ok as a backup solution. It can only hold 64 backups but that means over 2 months of data all backed up with a gui which a monkey could use.

 

It uses it's own form of dedupe so 64 backups = 1 full backup + everything which has changed over the 63 days/backup slots. Therefore 2 tb will probably be able to hold your file server for however long you want it to, plus all of your other servers too.

 

Licences are less than £30 per server which is just slightly cheaper than backup exec.

 

 

 

(Personally I like dhicks suggestion but relying on a system which you aren't 100% on in this situation is a bit risky)

[Vortex566]

Could I ask to see the spec please?

 

Do you run full backups each night? As Incremental would be alot better! We currently backup about 2.4TB each Friday to a custom built 28TB ubuntu server. Then each week night we backup the incremental stuff for the main file servers (full Sims Database copy though). If you want i can post the spec of the server which i bought (although if you want to run windows you would need to change the raid card as i run software raid.)

 

Overally it cost me circa 4K

[glennda]

ok thanks.

 

I didnt set up the backups, it was my predecessor. I like the idea of incremental backups. Would this work?

 

Monday - Full backup to NAS drive.

Tues -> Friday - Incremental backup to NAS.

End of month - Full backup to tape to store in safe.

 

Personally I would go with Full backup friday night (simply because mine tend to last longer then 12 hours or so and go on into the next day). Then monday-thurs incremental backups.

[glennda]

Is it just user files you'll be backing up, not VM images or anything like that? If so, why not use rsync to copy the files from your main file server every night to your backup server, a different backup folder every day, but with the backup server running some kind of deduplication system (either a deduplicating file system or a script that does file deduplication). That way you could probably hold more than a week's worth of week's worth of backups, depending on just how many files are changed per day.

 

can you use rsync on windows?

[glennda]

Could I ask to see the spec please?

 

Check this link.

 

You can ignore the HP microserver as that was used for replication of data.

 

we put 2 250gb drives in for the OS aswell in raid 1 (software raid) and then 16 x 2tb drives for the data which once formatted gives us 28/30tb i can't remember exactly.

 

Its a beast!

[cpjitservices]

Best way to go!!!!!

 

 

Check this link.

 

You can ignore the HP microserver as that was used for replication of data.

 

we put 2 250gb drives in for the OS aswell in raid 1 (software raid) and then 16 x 2tb drives for the data which once formatted gives us 28/30tb i can't remember exactly.

 

Its a beast!

[dhicks]

can you use rsync on windows?

 

Yes. I understand (although I've never used it) that some edition(s) of Windows Server have a block-level deduplicating file system (Single Instance Storage?). It should simply be a case of paying for the right edition of Windows Server (and possibly CALs) and rsyncing files to a different backup folder each night, the server should take care of the rest. You'd just need a script to delete older backups when disk space hit a certain threshold value.

[glennda]

Yes. I understand (although I've never used it) that some edition(s) of Windows Server have a block-level deduplicating file system (Single Instance Storage?). It should simply be a case of paying for the right edition of Windows Server (and possibly CALs) and rsyncing files to a different backup folder each night, the server should take care of the rest. You'd just need a script to delete older backups when disk space hit a certain threshold value.

 

Take a look at backupPC David, It does block level backups like you mention and is very clever in the way it stores the data. Yes it doesn't do full os backups but if a server completely dies a reinstall of windows (soon to be linux) is fairly straight forward.

[dhicks]

Take a look at backupPC David, It does block level backups like you mention and is very clever in the way it stores the data.

 

I think for block-level deduplication I'd prefer to have a file system that I could use to format a block device with and then mount as a SMB share. That way restoring user's files consists of simply pointing them at a read-only copy of their files on the backup server. File-level deduplication should work equally well, too.

[glennda]

I think for block-level deduplication I'd prefer to have a file system that I could use to format a block device with and then mount as a SMB share. That way restoring user's files consists of simply pointing them at a read-only copy of their files on the backup server. File-level deduplication should work equally well, too.

 

thats fair enough. I don't have the space to do that for the amount of data we have! even the 28tb would only keep a month of so's stuff in that case!

[dhicks]

thats fair enough. I don't have the space to do that for the amount of data we have! even the 28tb would only keep a month of so's stuff in that case!

 

No, that's the whole point of a deduplicating file system - you store replicated data once and have multiple pointers to it. At the block device / dedicated deduplicating file system level this means calculating a checksum on each block of data. This is the approach used by ZFS and a bunch of assorted FUSE-based filesystems available for Linux. Unfortuantly, I found ZFS for Linux rather unreliable, and I think development has now stalled, and the FUSE-based systems available are experimental and/or horribly slow. Therefore, the best approach is probably file-level deduplication, where you simply calculate a checksum on each individual file and replace duplicates with a hard link. This won't work for a live filesystem (because you might end up changing a file that you think is a separate file but is actually a hard link), but it's ideal for a read-only backup.

 

rsync seems to operate in a way that is exactly what you'd want to be compatible with the above system, i.e. it creates a whole new file when updating an existing file so it doesn't end up appending data to the linked file. Therefore, all you need is a very simple script that, every night, duplicates the previous day's backup folder into a new folder by duplicating the directory structure and creating hard links to files, then runs rsync to update any changes with your live file server. Each day you only use however much space it takes to duplicate the directory structure and space for changed files. I'm guessing your 28TB server could store daily backups of a 1TB file share going back several years.

[Willott]

Interestingly, I've built a backup box a little while ago - 8x2TB disks (2x80GB for OS) with space for 2 more disks if I need - total cost around £1.5k, though now would cost a lot more due to HDD prices. Case purchased off xcase (significantly cheaper case than the Supermicros, a little lower build quality, but does the job), low power hard drives (it's only a backup server, we're not expecting significant iops) and am hitting fairly decent speeds (about 80-90Mb/s continual across 1GB link). OS wise, grabbed NexentaStor Community Edition, though could've gone with Nexenta Core and would've done the job as well. Actually, just remembered I blogged about it - Willog » Terabytes on a budget… 2U 14.5TB usable backup device, although haven't updated that with OS HDDs (which increased the network throughput, USBs were causing some issues on some aspect of the system )

 

Depending on what exactly you're backing up, @dhicks idea is a good one, we're looking at backup software with capability to dedupe due to our backup arrangements. Expansion of dhicks idea could be to use inbuilt zfs snapshotting (so rsync just has to do update, you don't have it doing the move of older files), and possibly robocopy on windows (keep acls and use cifs for file transfers)