+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 17
Network and Classroom Management Thread, Trickle Backup in Technical; My Network Manager is after a backup program that will do trickle backup over the internet from an external site ...
  1. #1

    Join Date
    Apr 2008
    Posts
    405
    Thank Post
    22
    Thanked 13 Times in 10 Posts
    Rep Power
    15

    Trickle Backup

    My Network Manager is after a backup program that will do trickle backup over the internet from an external site to our main servers.

    Any ideas of which software would be worth a try?


    Thanks

  2. #2
    Jona's Avatar
    Join Date
    May 2007
    Location
    Cranleigh
    Posts
    467
    Thank Post
    14
    Thanked 50 Times in 48 Posts
    Rep Power
    23
    The key search term here is going to be differential or incremental backup. AFAIK all the major backup players do product that will do this. I've had success with a combination of smartsync pro and ftp but not in a production environment.....

    Cheers
    Jona

  3. #3

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,619
    Thank Post
    1,231
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by Reaper View Post
    Any ideas of which software would be worth a try?
    rsync.

    --
    David Hicks

  4. #4
    torledo's Avatar
    Join Date
    Oct 2007
    Posts
    2,928
    Thank Post
    168
    Thanked 155 Times in 126 Posts
    Rep Power
    47
    Quote Originally Posted by Reaper View Post
    My Network Manager is after a backup program that will do trickle backup over the internet from an external site to our main servers.

    Any ideas of which software would be worth a try?


    Thanks
    Never hears that term 'trickle' used before to describe a backup process..

    It's an apt description.

  5. #5
    monkeyx's Avatar
    Join Date
    Nov 2006
    Posts
    364
    Thank Post
    8
    Thanked 52 Times in 41 Posts
    Rep Power
    25
    Unison is also good, similar to Rsync.

  6. #6

    Join Date
    Apr 2008
    Posts
    405
    Thank Post
    22
    Thanked 13 Times in 10 Posts
    Rep Power
    15
    When he uses the term trickle I think he's referring to it being a slow process rather than a CPU intensive move. So as that way not to take up all the bandwidth, probably similar to WOWs update process that runs in the background.

    he also mentioned stuff like Cybervault and Mozy
    Last edited by Reaper; 11th April 2008 at 09:14 AM.

  7. #7
    steve's Avatar
    Join Date
    Oct 2005
    Location
    West Yorkshire
    Posts
    1,043
    Thank Post
    22
    Thanked 177 Times in 123 Posts
    Rep Power
    51
    Have a look a Symantec - Backup Exec Continuous Data Protection

    Only backs up changes to files, and runs all the time so bandwidth should be minimal - also allows bandwidth throttling.

  8. #8
    torledo's Avatar
    Join Date
    Oct 2007
    Posts
    2,928
    Thank Post
    168
    Thanked 155 Times in 126 Posts
    Rep Power
    47
    Quote Originally Posted by Reaper View Post
    When he uses the term trickle I think he's referring to it being a slow process rather than a CPU intensive move. So as that way not to take up all the bandwidth, probably similar to WOWs update process that runs in the background.

    he also mentioned stuff like Cybervault and Mozy
    If he's talkiing about a backup operation not overhelming a slow speed WAN link then the correct term for ensuring that doesn't happen is WAN optimization, specifically methods such as bandwidth throttling as mentioned by steve. I do like the idea of describing it as 'trickle' but it's not a known terminology.

  9. #9
    Joedetic's Avatar
    Join Date
    Jan 2006
    Location
    Walsall
    Posts
    1,316
    Thank Post
    6
    Thanked 13 Times in 13 Posts
    Rep Power
    22
    It's not an appropriate method for you to use for this, but something I keep meaning to take a look at is for Linux and it's DRBD (Distributed replication block device or something like that). Another machine on the network acts as a local block device and you can get it to replicate all blocks on the hard drive to the DRBD box which then takes regular snapshots and saves those so it's a very comprehensive sytem of live backup. (Essentially mirroring with archiving)

    In your situation I'd go with rsync. The initial sync will take a while over a WAN link though.

  10. #10

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,619
    Thank Post
    1,231
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by Joedetic View Post
    It's not an appropriate method for you to use for this, but something I keep meaning to take a look at is for Linux and it's DRBD
    Just got to having a look at this myself. I'm intending to use it with Xen - have writes to Xen virtual machine file systems mirrored to a separate physical machine for instant exploding-power-supply-proof failover. I'm having issues getting various bits to work together, though. Ubuntu 7.10 crashes on high I/O on our new (and very, very, cheap) Dell servers - something to do with glibc, I think. I'm using 7.04 instead. This runs Xen fine, but has issues with doing RAID 10 with mdadm - bizzare error necessitates reboot from install CD to fix broken system. And now that Xen is installed, the kernel header versions have changed so drdb won't install until I find linux-headers-2.6.19-4-server...

    --
    David Hicks

  11. #11
    Joedetic's Avatar
    Join Date
    Jan 2006
    Location
    Walsall
    Posts
    1,316
    Thank Post
    6
    Thanked 13 Times in 13 Posts
    Rep Power
    22
    Yeah it all looks like it could be quite complicated for what I'd want it for anyway..that and I'd need to look at locating a machine at the other end of the site connected by fibre or something so as not to pull huge load on the network.

  12. #12

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,619
    Thank Post
    1,231
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by Joedetic View Post
    Yeah it all looks like it could be quite complicated for what I'd want it for anyway..that and I'd need to look at locating a machine at the other end of the site connected by fibre or something so as not to pull huge load on the network.
    It should be less complicated if you're just cloning a file system - no Xen running means no messing around with header files and installing DRDB should just be a case of using apt-get to install the appropriate package. According to the DRDB documentation (which is very comprehensive and well-written, by the way) you can limit the bandwidth taken up by the replication system (I guess each written block is marked as "dirty" and shoved in a queue to be written when bandwidth allows?). Depends on how much data you have written to your server, too - I bet you don't actually get that much written, most requests will be reads.

    The way the DRDB documentation recommends linking two machines seems to be to simply plug them directly together via ethernet (I guess you could use a number of bonded ethernet ports). Skip the whole SAN business, you don't even need a switch to be able to have physically separated (well, okay, by up to 100m - technically wouldn't survive a nuclear bomb blast...) redundant-hardware systems capable of instant failover!

    I'm going to try re-installing Ubuntu 7.04 and this time compiling both Xen and DRDB from source to ensure I stick with the correct kernel headers. If this proves too much messing around then I'm switching to using just LVM volumes with periodic snapshots being used for backups.

    --
    David Hicks

  13. #13
    Joedetic's Avatar
    Join Date
    Jan 2006
    Location
    Walsall
    Posts
    1,316
    Thank Post
    6
    Thanked 13 Times in 13 Posts
    Rep Power
    22
    Well a company I had an interview a while ago were explaining about this being used for their backup process. They weren't using Xen but VMware on Gentoo. I had origionally wanted to learn to do this on FreeBSD but if it's going into the school where they'll have to support it when i'm gone i'm thinking maybe Debian/Ubuntu.

    Again...it's something I need to play with in VMs before I start in June.

    Out of interest (because I'm not entirely sure myself) what's the restore process? Just SCP the files back or something? But what if you've got to do an entire restore of a box?

  14. #14

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,619
    Thank Post
    1,231
    Thanked 776 Times in 673 Posts
    Rep Power
    235
    Quote Originally Posted by Joedetic View Post
    if it's going into the school where they'll have to support it when i'm gone i'm thinking maybe Debian/Ubuntu.
    I'd definatly wait a couple of weeks for Ubuntu 8.04 - Xen on 7.10 seems to have stability problems and mdadm on 7.04 is rather screwy too. Might want to see how you get on with Debian instead, or maybe Ubuntu 6.06 as it seems there's a seperate build of DRDB available for it.

    Out of interest (because I'm not entirely sure myself) what's the restore process? Just SCP the files back or something? But what if you've got to do an entire restore of a box?
    What, how do you restore a virtual machine, you mean? The idea is that DRDB keeps two filesystems exactly in synch, so you two exact copies on different machines. If one machine goes down then the other can take over. If you can't get the first machine back up again with the data intact then yes, I guess you'd simply have to copy the data over and start again.

    --
    David Hicks

  15. #15
    Joedetic's Avatar
    Join Date
    Jan 2006
    Location
    Walsall
    Posts
    1,316
    Thank Post
    6
    Thanked 13 Times in 13 Posts
    Rep Power
    22
    Well yeah, if you're using it to backup virtual machines...i was planning host OS backup...or doesn't it work like that?

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. OWA Backup
    By apeo in forum How do you do....it?
    Replies: 7
    Last Post: 4th December 2007, 01:47 PM
  2. Veritas Backup Exec Remote Backup
    By apeo in forum Windows
    Replies: 7
    Last Post: 20th September 2007, 10:16 PM
  3. Backup Tapes Replacement & Backup Schedules
    By robknowles in forum Hardware
    Replies: 2
    Last Post: 18th April 2007, 10:55 PM
  4. Replies: 9
    Last Post: 1st August 2006, 10:25 AM
  5. Backup Script for NT backup and Robocopy
    By ChrisH in forum Scripts
    Replies: 12
    Last Post: 20th October 2005, 01:01 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •