IIRC AMANDA (see Amanda Open Source Backup ) pretty much does this for you... have you checked it out before re-inventing the wheel?
I need a hand here, my scripting skills are insufficent to pull this one off. However the end result is likely to benifit others in the future. But first, some background info.
I have recently been creating a new file server running Ubuntu 8.04 on a HP DL140G3 with an external MSA60 Disk Array. Obviously to service the Windows clients this is running Samba (with all the ACL, quota, etc bells and whistles).
The underlying file system for the user storage is RAID-5. This is then allocated with LVM2 into various partitions. These are formatted as JFS and mounted some place sensible (/home/samba/blah normally) and then shared via samba as required.
I wish to use LVM based snapshots to provide 'shadow copies' to Windows clients. To do this I need a suite of scripts to do the following:
1) create LVM snapshots on a regular basis of some LVM paritions. New snapshots must be mounted in the way sambas shadow copy VFS module expects.
2) remount LVM snapshots cleanly in the case of a server reboot/crash/etc.
2) delete LVM snapshots after a predetermined time and/or when disk space is low.
A couple of links with background info:
Chapter23.Stackable VFS modules
Samba Shadow Copy Howto - Waikato Linux Users Group
Samba and MS Shadow Copy
I searched the site and only came up with a couple of articles mentioning snapshotting MySQL.
I have been and had a look... since I last looked at it a couple of years ago, a commercial arm has broken off and the web-interface features that I thought could have been used to solve your problem are only in the paid-for version.
Just write a script to read all the subdirectories in your mount directory and mount them all, then run that script from your rc.local?remount LVM snapshots cleanly in the case of a server reboot/crash/etc.
Just figure out the modified time of each subdirectory, compare it with the current date, delete it if you need to. For disk space, read all subdirectories, order by date, delete the first, loop until you have enough space.delete LVM snapshots after a predetermined time and/or when disk space is low.
How do you mean you're having difficulty writing a script for the above? Does it have to be done as a bash script, can't you simply write it in Perl or Python or something?
Plus there's no error handling.
So I need to be keeping track of which LVM2 volumes I wish to snapshot, and keep track of all their snapshots and the creation time of the snapshots.
Additionally, if the system has crashed, I need to fsck the volumes prior to mounting. Otherwise the mount fails.
Sorry, I meant as in how that example script works - it makes a new directory to stick mount points in, so you can assume that all subdirectories in that directory can be mounted.Er no, I have other volumes that are not on LVM2 (the raid1 arrays on the boot drive(s) for example) and I have LVM2 volumes that do not require shadow copy support. So I can't simply blindly assume every volume in /dev/disk/ or /dev/mapper/ is a LVM2 volume and/or snapshot.
Darned if I can remember how to do loops in bash, anyhow. Wouldn't a Python script, very roughly, be something along the lines of:I think this problem is too complex for a bash script. I was thinking Python, but as I said, my skills are limited.
Might want to replace the above calls to os.system with a function that prints the commands out before actually running them, letting you see what the script is going to do before actually doing it.Code:#!/usr/bin/python # Script to be run from a cron job, once a day, to create snapshots. import os import time volumes = ["volOne", "volTwo", "volEtc"] snapName = time.strftime("%Y.%m.%d-%H.%M.%S", time.gmtime()) for volume in volumes: os.system("xfs_freeze -f /data/shadow_share/") os.system("lvcreate -L10M -s -n "+volume+snapName+" /dev/shadowvol/sh_test") os.system("xfs_freeze -u /data/shadow_share/") os.system("mkdir /data/shadow_share/"+volume+snapName) os.system("mount /dev/shadowvol/"+volume+snapName+" /data/shadow_share/"+volume+snapName+" -onouuid,ro")
Last edited by dhicks; 31st July 2008 at 11:55 AM. Reason: Woops, need to change bash-style date/time to Python.
Well that's a start. I shall have a tinker and see how far I get. Thanks.
I'm might come across as a Solaris bigot . But if you want a filesystem that can do snapshots & CIFS/NFS/iSCSI support you must look at OpenSolaris ZFS.
The guys in the Hardware Forum rave on about Sun Unified Storage S7000 range, which is under the covers is basically OpenSolaris + BUI.
Snapshot are simple in ZFS and very efficient with space.
To access a snapshot just simply follow the hidden ".zfs" directory which access all the read only snapshots for the share.
EXAMPLE ONLY (Assuming zpool and Domain/Workgroup setup has been done)
Create snapshotCode:# Create CIFS share $ zfs create zpool/UserHomeDirs $ zfs set sharesmb=on zpool/UserHomeDirs
Create many CIFS shares and snapshot them in one commandCode:# Snapshot for Monday $ zfs snapshot zpool/UserHomeDirs@UserHomeDirs-Monday # Snapshot for Tuesday $ zfs snapshot zpool/UserHomeDirs@UserHomeDirs-Tuesday
I can't do ZFS functionality justice but it well worth the time researching OpenSolaris & ZFS. A blog that may be helpfull. A Home File server using ZFSCode:$ zfs create zpool/cifs-shares $ zfs set sharesmb=on zpool/cifs-shares $ zfs create zpool/cifs-shares/share1 $ zfs create zpool/cifs-shares/share2 ........ $ zfs create zpool/cifs-shares/share99 ## Snapshot shares 1 to 99 $ zfs snapshot -r zpool/cifs-shares@cifs-shares-Monday
There are currently 1 users browsing this thread. (0 members and 1 guests)