New Backup System Advice

[Cache]

Hi

 

I've recently started at a new school and the backup system which is currently in place (well, I reinstated since there wasn't one) has been outgrown I feel. (Having to start backing up on a Thursday to try and get the last tape in before I finish on Friday doesn't really seem ideal to me)

 

So what I thought of doing was putting together a proposal for a D2D2T system and wondered what your thoughts on what I had planned were?

 

Server

 

HP ProLiant ML110 G5 - Dual-Core Xeon 3065 2.33 GHz, 250GB SATA storage, 1GB RAM @ £399

HP Midline - Hard drive - 250 GB - internal - 3.5" - SATA-300 - 7200 rpm x3 @ £82.99 each

Windows Server 2008 Licence - Schools @ £56.80

 

Tape Library

 

Quantum Superloader 3 Tape Autoloader - LTO Ultrium @ £1521.99

LTO Ultrium Cleaning Cartridge @ £39.99

Sony LTO Ultrium 2 Media x 10 @ £21.99 each

 

Backup Exec

 

Backup Exec 12.5 with CPS for Windows Servers Licence @ £372

Backup Exec 12.5 Media @ £36

Backup Exec Windows Systems Agent x 3@ £223 each

 

Estimated Cost: £3344

 

Now, at the moment the system would need to backup 260GB on a full backup, however we will be getting a new SIMS server in shortly and potentially replacing one of the other servers over the next year, so I expect that to increase, however the question is does this go over the top. I'm not the budget holder and when asked roughly how much I could put towards this no figure has been mentioned so have just put down roughly what I found off various websites.

 

Any ideas what could be changed to potentially lower cost (as I'll probably be asked to)?

 

Cheers

[CyberNerd]

Replace this bit:

 

Windows Server 2008 Licence - Schools @ £56.80

 

Backup Exec 12.5 with CPS for Windows Servers Licence @ £372

Backup Exec 12.5 Media @ £36

Backup Exec Windows Systems Agent x 3@ £223 each

 

With this:

 

www.centos.org - The Community ENTerprise Operating System

rsback: pollux.franken.de

rsync

 

and you will save £1100

[Guest monkeyx]

We have not backed up to tapes for years now.

 

They tend to be slow and ureliable, for the money you are spending you build a disk backup system and store offsite backups on a USB drive?

 

We also use free/open source tools (Ununtu,ntbackup, volume shadow copy, unison and rsync) for backups, but they the key thing is backing up to disc, as much quicker and great reliability.

Edited May 11, 2009 by monkeyx

[DMcCoy]

Replace this bit:

 

 

 

With this:

 

www.centos.org - The Community ENTerprise Operating System

rsback: pollux.franken.de

rsync

 

and you will save £1100

 

Does rsync support windows ACLs and alternate data steams?

[DMcCoy]

for backups, but they the key thing is backing up to disc, as much quicker and great reliability.

 

I don't believe that this is true. My D2D is slower than my D2T, although I do D2D2T because the servers are not quick enough to keep up with the tape library. Are you encrypting any offsite copies too?

[CyberNerd]

Does rsync support windows ACLs and alternate data steams?

 

I use the windows NTbackup to backup the windows systems to a local disk, then backup that file with rsync. All our fileservers are linux now so I don't need to worry too much about backing up windows acls.

 

but I think rysnc should support it provided the the backup server was on the same domain, you could probably run rsync on the windows server and backup to an smb share - I guess this would keep perms ok. not tried it as samba seems to be a better fileserver for windows, than windows.

[Guest monkeyx]

I don't believe that this is true. My D2D is slower than my D2T, although I do D2D2T because the servers are not quick enough to keep up with the tape library. Are you encrypting any offsite copies too?

 

We run our back jobs in parallel to several disc systems, so yes it does run much faster. Most tape systems use the disc as a buffer, but you can emply the same principle for D2D2D

 

Yes we encrypt backup files for storage.

[Guest monkeyx]

not tried it as samba seems to be a better fileserver for windows, than windows.

 

About Samba being better than Windows for File and Print. I am big fan of linux but would never make such a claim. On what basis are you saying it is better cost, price or speed?

 

Have you ever had several hundred users writing to a Samba system at once, did it run faster then Windows SMB/CIFS?

Edited May 11, 2009 by monkeyx

[DMcCoy]

We run our back jobs in parallel to several disc systems, so yes it does run much faster. Most tape systems use the disc as a buffer, but you can emply the same principle for D2D2D

 

Yes we encrypt backup files for storage.

 

Running at 3500MB/min backup, 7000MB/min verify with encrypted LTO4 at the moment

 

It would backup faster but I can't get any more speed out of the crappy xraid box for reading, even with 7 disk RAID 0 setup.

[matt40k]

I stick with the rule of you backup to a different storage media, so if you use HDD to store the data, you backup to tape\dvd\zip.

 

You should also store a monthly tape which you store for at least 11 months and a yearly for, say, 5 years. (look on SupportNet for Capita, same sort of idea, but they say 3 months (i think) and you need to backup sat\sun - don't see the point unless you've got remote access that's heavly used).

 

You'll need to take this into account. Also, in the respect of virus spreading to your NAS (or whatever you want to call the blast thing), you're want to use another OS to the server your backing up, I recommend OpenBSD, finally, make the backup (partition\device) read only once you've created it. Basically if backup someones work, you don't want to be able to accidently corrupt, modify, delete, move it.

 

Make sure you log what you backup too, at least then you can say to teacher X "no you didn't save that document to your network drive, because if you did it would of got backed up, that's why I can't restore it." (normally to C:).

 

Back it up, encrypt it, keep it offsite, log it, document it.

[CyberNerd]

About Samba being better than Windows for File and Print. I am big fan of linux but would never make such a claim. On what basis are you saying it is better cost, price or speed?

 

Have you ever had several hundred users writing to a Samba system at once, did it run faster then Windows SMB/CIFS?

 

On the basis of it features. It was substantially more cost effective for us to migrate to Samba than windows 2003/2008 fileservers.

I've not benchmarked Samba's speed. We have 1800 users and 600 computers. All the student drives and the shared drives are on a Samba filesystem and we have no appreciable problems. Our samba install is certainly faster than older w2k servers and comparable with 2k3/2k8. Certainly not any slower, (especially if you consider that it's possible to set up multiple servers writing simultaneously to the same global filesystem....). IIRC current benchmarks put windows/samba on level pegging for speed, but historic benchmarks put Samba2 as twice the speed of win2k.

 

Contrary to popular belief smb fileserving isn't a microsoft invention - it was invented by IBM in the early 1990's and is used on lots of commercial unix and linux systems to deliver files to windows clients - I'm pretty certain it's at least as stable as any MS solution.

 

The main reason we use it is because we can:

 

1) veto certain filetypes, .exe .reg, .bat etc from existing on the filesystem

2) Have homedirectories (and permissions) automatically created when a user tries to access a share

3) Have a decent quota system, with soft/hard limits and grace periods, where quota violations get emailed to users and students can easily see their usage in windows.

4) Have a decent recycle bin facility so that we can recover files without going to backups.

5) Hide certain filetypes such as program config files

6) Remove the 'permissions' tab so that students can't even view file permissions, let alone change them.

7) easily search and run scripts on users homedirectories in bash/perl

8) actually hide shares, rather than 'dollar hide' shares.

9) is really very configurable and does lots of things that I might like to do if I knew how they worked.

 

I'm sure it's possible to do some of these things with windows and 3rd part addons - but why bother when a Centos/RHEL install does it all out of the box after a bit of configuration ?

[Guest monkeyx]

Contrary to popular belief smb fileserving isn't a microsoft invention - it was invented by IBM in the early 1990's and is used on lots of commercial unix and linux systems to deliver files to windows clients - I'm pretty certain it's at least as stable as any MS solution.

 

The main reason we use it is because we can:

 

1) veto certain filetypes, .exe .reg, .bat etc from existing on the filesystem

2) Have homedirectories (and permissions) automatically created when a user tries to access a share

3) Have a decent quota system, with soft/hard limits and grace periods, where quota violations get emailed to users and students can easily see their usage in windows.

4) Have a decent recycle bin facility so that we can recover files without going to backups.

5) Hide certain filetypes such as program config files

6) Remove the 'permissions' tab so that students can't even view file permissions, let alone change them.

7) easily search and run scripts on users homedirectories in bash/perl

8) actually hide shares, rather than 'dollar hide' shares.

9) is really very configurable and does lots of things that I might like to do if I knew how they worked.

 

I'm sure it's possible to do some of these things with windows and 3rd part addons - but why bother when a Centos/RHEL install does it all out of the box after a bit of configuration ?

 

Thanks for the history lesson re SMB I was using it along with NFS back in the 90s

 

But my historical perpective of SMB is part the problem. Also had some bad experiences with cheap NAS devices using old version of Samba. Will get cracking and re test use of Samba (volume shadow copy is a nice 2003 feature, 2003 r2 also finally allows for veto of file types!) as I more than happy to use Samba for File servers. In fact I would be delighted if we could so many thanks for the update

 

I no longer use Centos but Debian/Ubuntu hopefully samba should perform just as well on a non RedHat based distro.

 

Will report back on speed tests

[Cache]

We have not backed up to tapes for years now.

 

They tend to be slow and ureliable, for the money you are spending you build a disk backup system and store offsite backups on a USB drive?

 

We also use free/open source tools (Ununtu,ntbackup, volume shadow copy, unison and rsync) for backups, but they the key thing is backing up to disc, as much quicker and great reliability.

 

So do you just plug a usb HDD in and back it up to that then? Would you have one for each server and just use NT backup or still go with a separate backup server?

 

Will have to look in more detail at rsback/centos.

 

Thanks for the replies so far.

[Boon72]

wow this is nothing like we use at our schools, we use NT Backup to save backups to a usb hdd, daily incremental backups & 1 full weekly one, some schools swap them some don't.

[Guest monkeyx]

Well got a test Samba server running. Tested against Windows 2003 R2 on identical hardware.

 

The light user speed test showed that it was exactly the same speed, tried a few tweeks to make it faster but maybe that underlying hardware is the bottleneck Being the same speed is no bad thing though! Anyone know of any tweaks we could try?

 

Got the recycle bin working, but even better as we used LVM to hold our data, we are able to to full volume shadow copies (Documenting how this works as it was spread around across a few sites) linking to another LVM where they are held!

 

So very impressed with what we have found so far. Kicking myself for not using Samba for what it is designed for. The really sad part is that we used Samba heavily in Moodle and Squid for authenication etc!

[CyberNerd]

@monkeyX

good work. I recommend the linux quota system

Care to split this thread? I have some scritps that will sort out auto homdrivecreation

[Guest monkeyx]

Sure PM with new thread details.

 

I am working on creating a bash script to create snapshots LVMs in a seperate volume and then create an ln back to the orginal volume so that the snapshots can be seen.

[CyberNerd]

Sure PM with new thread details.

 

I am working on creating a bash script to create snapshots LVMs in a seperate volume and then create an ln back to the orginal volume so that the snapshots can be seen.

 

I added some details about our samba setup to the wiki:

 

http://www.edugeek.net/wiki/index.php/Samba_Homedirectory#enable_ACL_support

 

If you would be so kind as to include the LVM information

[Guest monkeyx]

Added some very rough details and script to http://www.edugeek.net/wiki/index.php/Samba_Homedirectory#Snap_Shotting_Volumes

 

Anyone have feedback on this ie more detail ?

[Cache]

Thanks for the replies all, it's been useful!

 

Been to put it on hold now until the budgets come through, but I'll revisit this thread once they do. It's given me more to think about anyway.

[Cache]

Slightly different tact now, as I really don't think I'm going to be able to get this sort of system in, so was wondering if anyone had used something like this: Insight UK - Tandberg RDX QuikStor - Disk drive - RDX - Hi-Sp... - 8550-RDX

 

Or would it be just as easy (and probably cheaper) to go and buy a load of usb HDD and back up to them?

 

Any suggestions?

 

Thanks

[Guest monkeyx]

One of our cheapest backup systems was to build a server(aka big PC!) and fill it with sata discs. We then setup hardware raid 5 (although software would be cheaper still). Ours runs Linux, but could just as easily run Windows.

[mattstevenson2005]

We are just having a new storage network installed and we have 2 SANs (2x Fujitsu SX80) that permanently replicate each other all the time, these are stored in two separate data centres on opposite ends of the school site then one of the SANs plug directly in to the fibre robotic tape library (Fujitsu TX24 LTO4) which can process up to 35Tb per backup.

[Guest monkeyx]

Just came across this bargain 1Tb USB drive for less than £70 Buffalo Drivestation 1TB External Drive TurboUSB With Power Saving Mode - Retail - Ebuyer

[cookie_monster]

I'm curious how people on here that just backup to disk manage to meet requirements for data protection archiving, some data needs to be kept for 7+years now. Also what about if you need to hunt down some files from two + years ago?

 

We currently use D2D2T and on a few occasions have had to restore data from a couple of year old archive tape. I have thought about using USB disks and just keeping one each year offsite but it's not as easy to keep track of what data is where as using tapes with backup exec for example.