A moodle on linux backup script

reggiep · June 18, 2009

I am having a go at making a script to automatically backup our moodle on linux and then run it with crontab. Can somebody take a look at my first effort and see if it could be improved?

I have blatantly reused code that was given to me by WEBMAN to backup zimbra. Thanks Webman.

#!/bin/bash

#
# Moodle Backup Script
#


# Variables
TIME=`date +%Y-%m-%d`
DOWN=`date +%u`
DOWD=`date +%A`

SOURCE="/var/moodlebackup/"
DEST="/tmp/moodlebackup"

NFSHOST="10.60.*.*"
NFSPATH="/mnt/array1/moodelbackup"
NFSMOUNT="/mnt/moodlebackup"

ARCHIVENAME="moodle-$DOWN-$DOWD.tar.gz"
ARCHIVE="/tmp/$ARCHIVENAME"

LOG="/var/log/moodlebackup.log"



function out {
echo "["`date +"%Y-%m-%d %T"`"] $1"
}


#echo ""
#echo "* * * * * * * * * *"
#echo "Moodle Backup"
#echo `date +%Y-%m-%d`
#echo ""


echo ""
out "Moodle Backup backup"
echo ""

tar -cvzpf $SOURCE /moodledata.tgz /var/moodledata
tar -cvzpf $SOURCE /moodlescripts.tgz /var/www/moodle
mysqldump -u root --password=password -C -Q -e -a moodle > /var/moodlebackup/moodle-backup.sql

echo ""
out "Creating archive"
echo ""

# Create archive of backed-up directory for transfer
tar -zcf $ARCHIVE -C $DEST .


echo ""
out "Copying to NFS server"
echo ""

mkdir $NFSMOUNT
mount -t nfs $NFSHOST:/$NFSPATH $NFSMOUNT
cp -fv $ARCHIVE $NFSMOUNT/
ls -lah $NFSMOUNT
umount $NFSMOUNT
rm -rfv $NFSMOUNT


echo ""
out "Removing $DEST and $ARCHIVE from local disk"
echo ""

# Remove temp backup destination, samba and archive
rm -rf $DEST
rm -fv $ARCHIVE

echo ""
out "Backup complete!"
echo ""
echo "----------"


cat $LOG | mail -c '' -s "[Moodle backup] `hostname --fqdn` $TIME" $EMAIL

CyberNerd · June 18, 2009

gpl license?

reggiep · June 18, 2009

You've lost me there cyberbnerd!

Are you asking me whether I should be putting the script up here without permission?

If so I don't know.

I could take it down again if Webman is not happy?

webman · June 18, 2009

More than happy - anyone is free to do with it what they want.

CyberNerd · June 18, 2009

you could prepend a GPL licence to it

reggiep · June 18, 2009

Firstly, you have lost me yet again cybernerd

Secondly I have changed it a biy now as it wasn't working for me but it is now!

Here it is again with alterations...

#!/bin/bash
#
#Moodle Backup Script
#

# Variables
TIME=`date +%Y-%m-%d`
DOWN=`date +%u`
DOWD=`date +%A`

DEST="/var/tempmoodlebackup"

NFSHOST="10.60.28.56"
NFSPATH="/mnt/array1/moodlebackup"
NFSMOUNT="/mnt/moodlebackup"

ARCHIVENAME="moodle-$DOWN-$DOWD.tar.gz"
ARCHIVE="/var/$ARCHIVENAME"

function out {
       echo "[" 'date +"%Y-%m-%d %T"'"] $1"
}


mkdir $DEST
tar -zcf $DEST/moodledata.tgz /var/moodledata
tar -zcf $DEST/moodlescripts.tgz /var/www/moodle
mysqldump -u root --password=password -C -Q -e -a moodle > $DEST/moodle-backup-full.sql


tar -zcf $ARCHIVE $DEST

mkdir $NFSMOUNT
mount -t nfs $NFSHOST:/$NFSPATH $NFSMOUNT
cp -fv $ARCHIVE $NFSMOUNT/
ls -lah $NFSMOUNT
umount $NFSMOUNT
rm -rfv $NFSMOUNT

echo ""
out "Removing $DEST and $ARCHIVE from local disk"
echo ""

rm -rf $DEST
rm -fv $ARCHIVE

echo ""
out "Backup Complete"
echo ""

Edited June 19, 2009 by reggiep

edtowens · May 5, 2011

Hi all, I would really like someone to look this over and tell me if they see some problems with security or potential problems with corruption. I have ran this script locally using ubuntu 10.10 using LAMP. Everything appears to work correctly. I have a vps and have successfully ran this script there also. I don't know what linux version my host is using but it seems to work. I place this script outside the webroot in a folder called Backups. This will create two backups only. If there are two backups already it will overwrite the oldest one. I didn't want to have a drive filled up with backups. You could alter it to create as many as you like though. I have created a restore script also that basically reverses the backup script.I left path information because it is just my local setup. Let me know what you think. If you want, I will also post my restore script.

#!/bin/bash
#==========================================================================================================================
# Script Name:	LocalMoodleBackup
# By:		Edward Owens
# Date:		April 2011
# Purpose:	Backup Moodle, Moodledata, and Database and zip them in a single zip file. Keep only two backups on hand. 
#               If there is already two backups on hand, then overwrite the oldest one.
#==========================================================================================================================

#------Variables-----------------------------------------------------------------------------------------------------------
suffix=$(date +%B-%Y)					# date stamp
logname="/home/edward/Backups/"$suffix"_backup.log"	# absolute path to logfile
file_name1="/home/edward/Backups/moodle1.zip"		# absolute path to backup 1
file_name2="/home/edward/Backups/moodle2.zip"		# absolute path to backup 2
zip1="/home/edward/webdev/domainname.com/"		# absolute path to moodle root
zip2="/home/edward/webdev/moodledata/"			# absolute path to moodle data
sql_name="/home/edward/Backups/moodle_db.sql"		# absolute path to sql file
dbName="my dbname"					# db Name
dbUser="my dbuser"					# db User
dbPassword="mypassword"					# db Password
dbHost="my dbhost"					# db Host
#----------------------------------------------------------------------------------------------------------------------------

echo "Started--> "$(date +%H":"%M":"%S) > $logname
echo "Dumping database"
mysqldump --opt --user=$dbUser --password=$dbPassword --host=$dbHost $dbName > $sql_name

if test -e $file_name1;
then
if test -e $file_name2;
then
	echo "copying moodle2.zip to moodle1.zip"
	cp $file_name2 $file_name1 >> $logname
	echo "removing moodle2.zip"
	rm $file_name2 >> $logname
	echo "creating moodle2.zip"
	zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
else
	echo "creating moodle2.zip"
	zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
fi
else
echo "creating moodle1.zip"
zip -r $file_name1 $zip1 $zip2 $dbname >> $logname
fi

# remove sql file
echo "removing "$sql_name
rm $sql_name
echo "Finished--> "$(date +%H":"%M":"%S) >> $logname
echo "Finished--> "$(date +%H":"%M":"%S)

Edited May 5, 2011 by edtowens
Make title more clear

reggiep · May 6, 2011

I wish I made the time to annotate scripts like you have edtowens!

Whenever i go back I always struggle with them.

You have spurred me on to go back through some of mine and tidy them up.

I would like to see your restore script as well please.

powdarrmonkey · May 6, 2011

Hi all, I would really like someone to look this over and tell me if they see some problems with security or potential problems with corruption. I have ran this script locally using ubuntu 10.10 using LAMP. Everything appears to work correctly. I have a vps and have successfully ran this script there also.

I don't know what linux version my host is using but it seems to work.

You can get some clues, try:

$ cat /etc/debian_version
$ cat /etc/redhat-release
$ uname -a

(which host?)

if test -e $file_name1;
then
if test -e $file_name2;
then
	echo "copying moodle2.zip to moodle1.zip"
	cp $file_name2 $file_name1 >> $logname
	echo "removing moodle2.zip"
	rm $file_name2 >> $logname
	echo "creating moodle2.zip"
	zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
else
	echo "creating moodle2.zip"
	zip -r $file_name2 $zip1 $zip2 $dbname >> $logname
fi
else
echo "creating moodle1.zip"
zip -r $file_name1 $zip1 $zip2 $dbname >> $logname
fi

# remove sql file
echo "removing "$sql_name
rm $sql_name
echo "Finished--> "$(date +%H":"%M":"%S) >> $logname
echo "Finished--> "$(date +%H":"%M":"%S)

Never trust your environment! You should do things like:

- test that the files you want to write are not directories and you have write permissions (see File test operators)

- specify the full path to executables, don't rely on PATH (see the man page for which(1) if you're not sure)

- check executables are really executable, before executing them

- fail on error! Don't just blindly carry on, use STDERR and STDOUT to report errors appropriately and exit()

Make your script portable (avoid using bashisms) and run it with /bin/sh, in case the user has another preferred shell.

What if the disk in the server fails - do you copy your backups to somewhere geographically remote? Do you compute any checksums to check they copied correctly?

Does somebody check that it happened and there was nothing unexpected - the backup shrank by 1GB for example? If you use STDERR/STDOUT and cron properly this is a three-second job in the morning.

edtowens · May 6, 2011

I would like to see your restore script as well please.

Thank you for your response reggiep. I will get that restore script up in a couple of days. The reason I annotate my scripts is because I'm still learning and it makes it easier for me to go over and edit. Plus, when I share it it will help someone else to modify it. I too struggle with linux and scripts, but I'm finding it much fun!

Thanks,

Edward Owens

edtowens · May 6, 2011

Hi powdarrmonkey

You can get some clues, try:

$ cat /etc/debian_version
$ cat /etc/redhat-release
$ uname -a

I will definitely do this.

(which host?)

Dreamhost

Never trust your environment! You should do things like:

- test that the files you want to write are not directories and you have write permissions (see File test operators)
- specify the full path to executables, don't rely on PATH (see the man page for which(1) if you're not sure)
- check executables are really executable, before executing them
- fail on error! Don't just blindly carry on, use STDERR and STDOUT to report errors appropriately and exit()

Nice advice and I thank you for it!

Make your script portable (avoid using bashisms) and run it with /bin/sh, in case the user has another preferred shell.

Since I'm a noob, I'm not sure what you mean by this.

What if the disk in the server fails - do you copy your backups to somewhere geographically remote? Do you compute any checksums to check they copied correctly?

I log in FTP and download the backups every other day. I will look into better file testing methods.

Does somebody check that it happened and there was nothing unexpected - the backup shrank by 1GB for example? If you use STDERR/STDOUT and cron properly this is a three-second job in the morning.

Honestly, never even thought about, but now I will!

Thanks,

Edward Owens

powdarrmonkey · May 6, 2011

Dreamhost

They reckon it's Debian - good choice!

(of course, with my Debian Developer hat on I would say that... )

Since I'm a noob, I'm not sure what you mean by this.

https://wiki.ubuntu.com/DashAsBinSh

edtowens · May 10, 2011

I would like to see your restore script as well please.

Here it is reggiep, sorry it took me a little while to throw it up for ya. It doesn't have the fixes that powdarmonkey had advised yet. I will try and make those revisions and post them back on this forum.

#!/bin/bash
#==========================================================================================================================
# Script Name:	MoodleRestore			 *Local*
# By:		Edward Owens
# Date:		May 2011
# Purpose:	Restore Moodle Root, Moodledata, and Database from the argument of 1 or 2.
#               MoodleRestore 1 will restore moodle1.zip, MoodleRestore 2 will restore moodle2.zip
#		This works in tandem with the Backup script I have created, 'MoodleBackup'.
#==========================================================================================================================

# define variables
zip1="/home/edward/Backups/moodle1.zip" 	# absolute path to backupfile 1
zip2="/home/edward/Backups/moodle2.zip" 	# absolute path to backupfile 2
lbesql="/home/edward/Backups/moodle_db.sql" 	# absolute path to sql file after extraction
logname="/home/edward/Backups/restore_log"	# absolute path to restore log
dbPassword="password"				# mysql password
dbUser="user"					# mysql user
DBNAME="databasename"				# mysql db name


DBEXISTS=$(mysql --batch --skip-column-names -e "SHOW DATABASES LIKE '"$DBNAME"';" | grep "$DBNAME" > /dev/null; echo "$?") # does database exist?

if [ $DBEXISTS -eq 0 ];then
echo "Dropping database because it exists"
mysqladmin -u[$dbUser] -p[$dbPassword] drop [$DBNAME]   
fi

if [ "$1" -eq "1" ]; # Want to restore backup file 1
then
if test -e $zip1; # Does backup file 1 exist?
then
	unzip -o $zip1 -d "/" >> $logname
	mysql -u $dbUser -p -D$DBNAME --password=$dbPassword < $lbesql 
	rm $lbesql
else
	echo $zip1" does not exist. Make sure you run MoodleBackup first!"
	exit
fi
else
if [ "$1" -eq "2" ]; # Want to restore backup file 2
then	
	if test -e $zip2; # Does backup file 2 exist?
	then
		unzip -o $zip2 -d "/" >> $logname
		mysql -u $dbUser -p -D$DBNAME --password=$dbPassword < $lbesql
		rm $lbesql
	else
		echo $zip2" does not exist. Make sure you run MoodleBackup first!"
		exit
	fi
else
	echo "No file selected"
fi
fi

fcortes · June 8, 2015

Hello

I found this thread and ed's script recently and I managed to adapt it and used it (to backup a moodle install) so first of all thank you Ed for your effort there.

now a question (and an observation) that I've been meaning to ask on the script to backup the moodle install (and I apologize beforehand as I'm pretty much a noob when it comes to scripting so pardon my ignorance on that subject):

when you say:

zip -r $file_name2 $zip1 $zip2 $dbname >> $logname

shouldn't $dbname be $sql_name instead?

since I assume that the line is zip your moodle folder, data folder and your db dump file (and not the name of the db) ?

Thanks

Edited June 8, 2015 by fcortes

A moodle on linux backup script

Recommended Posts

reggiep

CyberNerd

reggiep

webman

CyberNerd

reggiep

edtowens

reggiep

powdarrmonkey

edtowens

edtowens

powdarrmonkey

edtowens

fcortes

Create an account or sign in to comment

Create an account

Sign in

Recent Threads

Sage 200 budget holders - test something for me?

Bromcom Support

My Fluke LinkRunner has a fault...

Home

Forums

What's New