Disk based backups

[katie.lawe]

Hi

 

I am looking at implementing a disk based backup system, possibly qnap or something similar.

 

I have been reading lots of different ways of doing backups, what I want to know is can you run a system through a disk based backup like a tape system, eg keeping backup sets on a weekly cycle etc, instead of it overwriting/replicating the same files each night?

 

Thanks

[User3204]

It depends on the software.

 

But I'd go with a 'yes', there'll be a way to make whatever software do it.

[twin--turbo]

We use Microsoft DPM, it does disk based backups at 8hr Intervals and a Tape backup every Friday.

 

Does BMR and system state too.

 

We do have the advantage that our disk array for the backup is 5 miles away so a catastrophic event on site is unlikely to destroy the backup too.

 

Rob

[dhicks]

I am looking at implementing a disk based backup system, possibly qnap or something similar.

 

Hardware-wise, we've found the QNAP system we have is okay, but FreeNAS offers more features, including proper ZFS block-level de-duplication. QNAP's hardware doesn't seem to be anything that special (the motherboard inside the box seems to consist of an Atom processor with on-board SATA ports), and it would be cheaper to buy a Core i3/i5 system to install FreeNAS on (bear in mind you'll need a decent amount of RAM, 8GB minimum probably, to get proper usage out of the deduplication features). I would go for something like Tranquil PC's new Riley server - Core i5, 8GB RAM, fanless and completly silent so you could place them anywhere in the school without disturbing anyone.

 

what I want to know is can you run a system through a disk based backup like a tape system, eg keeping backup sets on a weekly cycle etc, instead of it overwriting/replicating the same files each night?

 

The point of a disk-based backup system is that it is one large pool of storage, not a linear device like a tape. As long as you have the file you want somewhere on the disk you can store a "copy" of it by simply making a hard link to that file, or if your system supports block-level deduplication just copy the file and let the system take care of sorting out the disk space. Depending on how much data your users change every day, you should be able to keep versions of your file system go back at least several weeks, if not months or years.

 

There really is no need to buy software to do backups, all you need to do is either copy updated files from one filesystem to another or copy block devices (virtual machine harddrive images, generally) around. Copying files can be done with your system's inbuilt utilities (copy, xcopy, robocopy, etc) or rsync and block devices are easy enough to read/write in most scripting languages.

[tmcd35]

We use Barracuda's Yosemite software to do just this (used to be known as Tapeware). It creates "virtual tape drives" on the NAS which it then backs up to as if it was another tape drive - albeit a lot faster.

 

We are using Buffalo TerraStations at the moment. The Windows Storage Server versions are interesting with iSCSI support. The other NAS boxes to catch my eye are the Netgear ReadyNAS range which look very interesting from a de-dupe point of view.

[twin--turbo]

Oh...

 

Just a P.S. on my DPM ramble, I hate it.

 

Been meaning to see if our old BackupExec does it any better.

 

Rob

[localzuk]

Hardware-wise, we've found the QNAP system we have is okay, but FreeNAS offers more features, including proper ZFS block-level de-duplication. QNAP's hardware doesn't seem to be anything that special (the motherboard inside the box seems to consist of an Atom processor with on-board SATA ports), and it would be cheaper to buy a Core i3/i5 system to install FreeNAS on (bear in mind you'll need a decent amount of RAM, 8GB minimum probably, to get proper usage out of the deduplication features). I would go for something like Tranquil PC's new Riley server - Core i5, 8GB RAM, fanless and completly silent so you could place them anywhere in the school without disturbing anyone.

 

The point of a disk-based backup system is that it is one large pool of storage, not a linear device like a tape. As long as you have the file you want somewhere on the disk you can store a "copy" of it by simply making a hard link to that file, or if your system supports block-level deduplication just copy the file and let the system take care of sorting out the disk space. Depending on how much data your users change every day, you should be able to keep versions of your file system go back at least several weeks, if not months or years.

 

There really is no need to buy software to do backups, all you need to do is either copy updated files from one filesystem to another or copy block devices (virtual machine harddrive images, generally) around. Copying files can be done with your system's inbuilt utilities (copy, xcopy, robocopy, etc) or rsync and block devices are easy enough to read/write in most scripting languages.

 

All of that is fine if you want to take all responsibility for the system onto your shoulders. However, disk based backup is not just how you say. It is used as a 'library' by many just like tapes were. An example of such a requirement would be a yearly backup for finance data.

 

I use BackupAssist to do the job, backing up to a QNAP device. Performance is pretty good, not perfect, but good.

[dhicks]

However, disk based backup is not just how you say. It is used as a 'library' by many just like tapes were. An example of such a requirement would be a yearly backup for finance data.

 

Sorry, I don't quite understand what you mean. Given sufficient storage space, you could store a snapshot of your data every day for as long back as you like. Once you start to run low on space you start removing backup copies, but you don't have to start with the earliest, you could delete every other day, or leave a once-a-quater copy for copies older than a year, or whatever interval you choose. Block device images are a bit more awkward as you'd want to store just the changes made, so you'd probably store a a diff file every time you did a backup. That could start making it a bit slow to restore images - restore the image from 3 days ago and you have to apply the most recent diff, then the one before that, then the one before that, etc. Of course, ZFS would seem to make this really easy - just set a snapshot point and sync your backup copy of a block device with the live one, no need to worry about snapshots degrading performance (if I understand how ZFS works correctly, in comparison with Linux's LVM which gets slower with each snapshot you apply).

[localzuk]

Sorry, I don't quite understand what you mean. Given sufficient storage space, you could store a snapshot of your data every day for as long back as you like. Once you start to run low on space you start removing backup copies, but you don't have to start with the earliest, you could delete every other day, or leave a once-a-quater copy for copies older than a year, or whatever interval you choose. Block device images are a bit more awkward as you'd want to store just the changes made, so you'd probably store a a diff file every time you did a backup. That could start making it a bit slow to restore images - restore the image from 3 days ago and you have to apply the most recent diff, then the one before that, then the one before that, etc. Of course, ZFS would seem to make this really easy - just set a snapshot point and sync your backup copy of a block device with the live one, no need to worry about snapshots degrading performance (if I understand how ZFS works correctly, in comparison with Linux's LVM which gets slower with each snapshot you apply).

 

One of the requirements that was placed on my backup solution at my last place was that every year, all admin data must be frozen and stored as an individual 'full' backup. So that if an auditor comes in and demands something, we could go 'here is the backup from that year, frozen'.

[dhicks]

One of the requirements that was placed on my backup solution at my last place was that every year, all admin data must be frozen and stored as an individual 'full' backup.

 

That's what a snapshot is - if you're used to LVM snapshots, where you generally take a snapshot and dump the data somewhere else as quickly as possible so you can remove the snapshot before performance degrades too badly, then ZFS snapshots are snapshots done right. You set a snapshot and that's your backup, the disk state frozen in time, ready to mount as a virtual machine disk if needed. Of course, if you wanted you could copy your yearly backup to a separate, removeable disk for placing in a fire safe, just for added safety.