A moodle on linux backup script

[reggiep]

I am having a go at making a script to automatically backup our moodle on linux and then run it with crontab. Can somebody take a look at my first effort and see if it could be improved?
I have blatantly reused code that was given to me by WEBMAN to backup zimbra. Thanks Webman.

#!/bin/bash

#
# Moodle Backup Script
#


# Variables
TIME=`date +%Y-%m-%d`
DOWN=`date +%u`
DOWD=`date +%A`

SOURCE="/var/moodlebackup/"
DEST="/tmp/moodlebackup"

NFSHOST="10.60.*.*"
NFSPATH="/mnt/array1/moodelbackup"
NFSMOUNT="/mnt/moodlebackup"

ARCHIVENAME="moodle-$DOWN-$DOWD.tar.gz"
ARCHIVE="/tmp/$ARCHIVENAME"

LOG="/var/log/moodlebackup.log"



function out {
	echo "["`date +"%Y-%m-%d %T"`"] $1"
}


#echo ""
#echo "* * * * * * * * * *"
#echo "Moodle Backup"
#echo `date +%Y-%m-%d`
#echo ""


echo ""
out "Moodle Backup backup"
echo ""

tar -cvzpf $SOURCE /moodledata.tgz /var/moodledata
tar -cvzpf $SOURCE /moodlescripts.tgz /var/www/moodle
mysqldump -u root --password=password -C -Q -e -a moodle > /var/moodlebackup/moodle-backup.sql

echo ""
out "Creating archive"
echo ""

# Create archive of backed-up directory for transfer
tar -zcf $ARCHIVE -C $DEST .


echo ""
out "Copying to NFS server"
echo ""

mkdir $NFSMOUNT
mount -t nfs $NFSHOST:/$NFSPATH $NFSMOUNT
cp -fv $ARCHIVE $NFSMOUNT/
ls -lah $NFSMOUNT
umount $NFSMOUNT
rm -rfv $NFSMOUNT


echo ""
out "Removing $DEST and $ARCHIVE from local disk"
echo ""

# Remove temp backup destination, samba and archive
rm -rf $DEST
rm -fv $ARCHIVE

echo ""
out "Backup complete!"
echo ""
echo "----------"


cat $LOG | mail -c '' -s "[Moodle backup] `hostname --fqdn` $TIME" $EMAIL

[CyberNerd]

gpl license?

[reggiep]

You've lost me there cyberbnerd!

Are you asking me whether I should be putting the script up here without permission?

If so I don't know.

I could take it down again if Webman is not happy?

[webman]

More than happy - anyone is free to do with it what they want.

[CyberNerd]

you could prepend a GPL licence to it

[reggiep]

Firstly, you have lost me yet again cybernerd

Secondly I have changed it a biy now as it wasn't working for me but it is now!

Here it is again with alterations...

#!/bin/bash
#
#Moodle Backup Script
#

# Variables
TIME=`date +%Y-%m-%d`
DOWN=`date +%u`
DOWD=`date +%A`

DEST="/var/tempmoodlebackup"

NFSHOST="10.60.28.56"
NFSPATH="/mnt/array1/moodlebackup"
NFSMOUNT="/mnt/moodlebackup"

ARCHIVENAME="moodle-$DOWN-$DOWD.tar.gz"
ARCHIVE="/var/$ARCHIVENAME"

function out {
        echo "[" 'date +"%Y-%m-%d %T"'"] $1"
}


mkdir $DEST
tar -zcf $DEST/moodledata.tgz /var/moodledata
tar -zcf $DEST/moodlescripts.tgz /var/www/moodle
mysqldump -u root --password=password -C -Q -e -a moodle > $DEST/moodle-backup-full.sql


tar -zcf $ARCHIVE $DEST

mkdir $NFSMOUNT
mount -t nfs $NFSHOST:/$NFSPATH $NFSMOUNT
cp -fv $ARCHIVE $NFSMOUNT/
ls -lah $NFSMOUNT
umount $NFSMOUNT
rm -rfv $NFSMOUNT

echo ""
out "Removing $DEST and $ARCHIVE from local disk"
echo ""

rm -rf $DEST
rm -fv $ARCHIVE

echo ""
out "Backup Complete"
echo ""

[edtowens]

Hi all, I would really like someone to look this over and tell me if they see some problems with security or potential problems with corruption. I have ran this script locally using ubuntu 10.10 using LAMP. Everything appears to work correctly. I have a vps and have successfully ran this script there also. I don't know what linux version my host is using but it seems to work. I place this script outside the webroot in a folder called Backups. This will create two backups only. If there are two backups already it will overwrite the oldest one. I didn't want to have a drive filled up with backups. You could alter it to create as many as you like though. I have created a restore script also that basically reverses the backup script.I left path information because it is just my local setup. Let me know what you think. If you want, I will also post my restore script.

#!/bin/bash
#==========================================================================================================================
# Script Name:	LocalMoodleBackup
# By:		Edward Owens
# Date:		April 2011
# Purpose:	Backup Moodle, Moodledata, and Database and zip them in a single zip file. Keep only two backups on hand. 
#               If there is already two backups on hand, then overwrite the oldest one.
#==========================================================================================================================

#------Variables-----------------------------------------------------------------------------------------------------------
suffix=$(date +%B-%Y)					# date stamp
logname="/home/edward/Backups/"$suffix"_backup.log"	# absolute path to logfile
file_name1="/home/edward/Backups/moodle1.zip"		# absolute path to backup 1
file_name2="/home/edward/Backups/moodle2.zip"		# absolute path to backup 2
zip1="/home/edward/webdev/domainname.com/"		# absolute path to moodle root
zip2="/home/edward/webdev/moodledata/"			# absolute path to moodle data
sql_name="/home/edward/Backups/moodle_db.sql"		# absolute path to sql file
dbName="my dbname"					# db Name
dbUser="my dbuser"					# db User
dbPassword="mypassword"					# db Password
dbHost="my dbhost"					# db Host
#----------------------------------------------------------------------------------------------------------------------------

echo "Started--> "$(date +%H":"%M":"%S) > $logname
echo "Dumping database"
mysqldump --opt --user=$dbUser --password=$dbPassword --host=$dbHost $dbName > $sql_name

if test -e $file_name1;
then
	if test -e $file_name2;
	then
		echo "copying moodle2.zip to moodle1.zip"
		cp $file_name2 $file_name1 >> $logname
		echo "removing moodle2.zip"
		rm $file_name2 >> $logname
		echo "creating moodle2.zip"
		zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
	else
		echo "creating moodle2.zip"
		zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
	fi
else
	echo "creating moodle1.zip"
	zip -r $file_name1 $zip1 $zip2 $dbname >> $logname
fi

# remove sql file
echo "removing "$sql_name
rm $sql_name
echo "Finished--> "$(date +%H":"%M":"%S) >> $logname
echo "Finished--> "$(date +%H":"%M":"%S)

[reggiep]

I wish I made the time to annotate scripts like you have edtowens!
Whenever i go back I always struggle with them.
You have spurred me on to go back through some of mine and tidy them up.

I would like to see your restore script as well please.

[powdarrmonkey]

Hi all, I would really like someone to look this over and tell me if they see some problems with security or potential problems with corruption. I have ran this script locally using ubuntu 10.10 using LAMP. Everything appears to work correctly. I have a vps and have successfully ran this script there also.

I don't know what linux version my host is using but it seems to work.

You can get some clues, try:

$ cat /etc/debian_version
$ cat /etc/redhat-release
$ uname -a

(which host?)

if test -e $file_name1;
then
	if test -e $file_name2;
	then
		echo "copying moodle2.zip to moodle1.zip"
		cp $file_name2 $file_name1 >> $logname
		echo "removing moodle2.zip"
		rm $file_name2 >> $logname
		echo "creating moodle2.zip"
		zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
	else
		echo "creating moodle2.zip"
		zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
	fi
else
	echo "creating moodle1.zip"
	zip -r $file_name1 $zip1 $zip2 $dbname >> $logname
fi

# remove sql file
echo "removing "$sql_name
rm $sql_name
echo "Finished--> "$(date +%H":"%M":"%S) >> $logname
echo "Finished--> "$(date +%H":"%M":"%S)

Never trust your environment! You should do things like:

- test that the files you want to write are not directories and you have write permissions (see File test operators)
- specify the full path to executables, don't rely on PATH (see the man page for which(1) if you're not sure)
- check executables are really executable, before executing them
- fail on error! Don't just blindly carry on, use STDERR and STDOUT to report errors appropriately and exit()

Make your script portable (avoid using bashisms) and run it with /bin/sh, in case the user has another preferred shell.

What if the disk in the server fails - do you copy your backups to somewhere geographically remote? Do you compute any checksums to check they copied correctly?

Does somebody check that it happened and there was nothing unexpected - the backup shrank by 1GB for example? If you use STDERR/STDOUT and cron properly this is a three-second job in the morning.

[edtowens]

I would like to see your restore script as well please.

Thank you for your response reggiep. I will get that restore script up in a couple of days. The reason I annotate my scripts is because I'm still learning and it makes it easier for me to go over and edit. Plus, when I share it it will help someone else to modify it. I too struggle with linux and scripts, but I'm finding it much fun!

Thanks,

Edward Owens

[edtowens]

Hi powdarrmonkey

You can get some clues, try:

$ cat /etc/debian_version
$ cat /etc/redhat-release
$ uname -a

I will definitely do this.

(which host?)

Dreamhost

Never trust your environment! You should do things like:

- test that the files you want to write are not directories and you have write permissions (see File test operators)
- specify the full path to executables, don't rely on PATH (see the man page for which(1) if you're not sure)
- check executables are really executable, before executing them
- fail on error! Don't just blindly carry on, use STDERR and STDOUT to report errors appropriately and exit()

Nice advice and I thank you for it!

Make your script portable (avoid using bashisms) and run it with /bin/sh, in case the user has another preferred shell.

Since I'm a noob, I'm not sure what you mean by this.

What if the disk in the server fails - do you copy your backups to somewhere geographically remote? Do you compute any checksums to check they copied correctly?

I log in FTP and download the backups every other day. I will look into better file testing methods.

Does somebody check that it happened and there was nothing unexpected - the backup shrank by 1GB for example? If you use STDERR/STDOUT and cron properly this is a three-second job in the morning.

Honestly, never even thought about, but now I will!

Thanks,


Edward Owens

[powdarrmonkey]

Dreamhost

They reckon it's Debian - good choice!

(of course, with my Debian Developer hat on I would say that... )

Since I'm a noob, I'm not sure what you mean by this.

https://wiki.ubuntu.com/DashAsBinSh

[edtowens]

I would like to see your restore script as well please.

Here it is reggiep, sorry it took me a little while to throw it up for ya. It doesn't have the fixes that powdarmonkey had advised yet. I will try and make those revisions and post them back on this forum.

#!/bin/bash
#==========================================================================================================================
# Script Name:	MoodleRestore			 *Local*
# By:		Edward Owens
# Date:		May 2011
# Purpose:	Restore Moodle Root, Moodledata, and Database from the argument of 1 or 2.
#               MoodleRestore 1 will restore moodle1.zip, MoodleRestore 2 will restore moodle2.zip
#		This works in tandem with the Backup script I have created, 'MoodleBackup'.
#==========================================================================================================================

# define variables
zip1="/home/edward/Backups/moodle1.zip" 	# absolute path to backupfile 1
zip2="/home/edward/Backups/moodle2.zip" 	# absolute path to backupfile 2
lbesql="/home/edward/Backups/moodle_db.sql" 	# absolute path to sql file after extraction
logname="/home/edward/Backups/restore_log"	# absolute path to restore log
dbPassword="password"				# mysql password
dbUser="user"					# mysql user
DBNAME="databasename"				# mysql db name


DBEXISTS=$(mysql --batch --skip-column-names -e "SHOW DATABASES LIKE '"$DBNAME"';" | grep "$DBNAME" > /dev/null; echo "$?") # does database exist?

if [ $DBEXISTS -eq 0 ];then
	echo "Dropping database because it exists"
	mysqladmin -u[$dbUser] -p[$dbPassword] drop [$DBNAME]   
fi

if [ "$1" -eq "1" ]; # Want to restore backup file 1
then
	if test -e $zip1; # Does backup file 1 exist?
	then
		unzip -o $zip1 -d "/" >> $logname
		mysql -u $dbUser -p -D$DBNAME --password=$dbPassword < $lbesql 
		rm $lbesql
	else
		echo $zip1" does not exist. Make sure you run MoodleBackup first!"
		exit
	fi
else
	if [ "$1" -eq "2" ]; # Want to restore backup file 2
	then	
		if test -e $zip2; # Does backup file 2 exist?
		then
			unzip -o $zip2 -d "/" >> $logname
			mysql -u $dbUser -p -D$DBNAME --password=$dbPassword < $lbesql
			rm $lbesql
		else
			echo $zip2" does not exist. Make sure you run MoodleBackup first!"
			exit
		fi
	else
		echo "No file selected"
	fi
fi