Effective Moodle Backups

[Gatt]

Ok i have a cronjob that backups our Moodle db every night, but I'm looking for the best way to rsync our moodledata directory?

[CyberNerd]

but I'm looking for the best way to rsync our moodledata directory?

 

We use rsback Error! and pick up the moodledata (and db dump) onto a remote server.

To keep it simple you could just use rsync, or rsync over ssh to be more secure.

[Gatt]

That links giving me an error about username & password ?

trying rsync to our TeraStation at present

just not sure how often I should do it or hjow much backup history to keep

[CyberNerd]

we just use rsback in a nightly cron job, logs of the cron get emailed to me nightly.

The rsback scripts (when the website is working) provide a way of hardlinking files to save diskspace and a good way to rotate the backups.

 

here;s googes cache

 

rsback: pollux.franken.de

[BishopVLE]

How do you back up your database?

[Geoff]

I just cronjob a mysqldump into a folder. Then let the file system backup pickup the dumps.

[Gatt]

I do the same as Geoff for the mysql - its backing up the moodledata folder

think I have rsync sussed, but its a backup schedule I need to work out as the folder is currently <800MB and growing!

 

Do I just overwrite it each time or keep a weekly/monthly copy of it?

[CyberNerd]

rsync will only copy the file if it has changed. if you use the --delete option it will remove from backup anything that has been deleted from the original. The rsback scripts that I posted will deal with these problems and will save disk space because they snapshot each previous backup.

Alternatively, just do a nightly backup and put it onto tape once a week.

[BishopVLE]

How safe is MysqlDump with the server running?

 

If I use this

 

# Stop MySQLdatabase
/etc/init.d/mysql stop
echo `date + \%d/%m/%y.%H:%M:%S ` >> $DATABASEBACKUPLOG
echo 'Stopping MySQL database.' >> $DATABASEBACKUPLOG

#Dump new files
for i in $(echo 'SHOW DATABASES;' | $MYSQL -u$MYSQLUSER -p$MYSQLPWD -h$MYSQLHOST|$GREP -v '^Database$'); do
 $MYSQLDUMP                                                    \
 -u$MYSQLUSER -p$MYSQLPWD -h$MYSQLHOST                         \
 -Q -c -C --add-drop-table --add-locks --quick --lock-tables   \
 $i > $MYSQLBACKUPDIR/$i.sql;
done;

#Start MySQLDatabase
/etc/init.d/mysql restart

 

It won't work, unless I comment out the start and stop the database.

So is it ok to dump the database with it running? Or how do i do it with it stopping and starting

[Gatt]

It appears to be - I do it all the time!

my Cron script looks like this:

 

 
#/bin/sh
/usr/bin/mysqldump -u<> -p<> --opt moodle > /mysql_backups/moodle/moodle.sql
/usr/bin/gzip /mysql_backups/moodle/moodle.sql
mkdir -p /mysql_backups/moodle/`/bin/date +%Y/%b`
/bin/mv /mysql_backups/moodle/moodle.sql.gz /mysql_backups/moodle/`/bin/date +%Y/%b/moodle.sql_%d-%b-%Y.gz`

 

This creates a folder path as such:

 

/mysql_backups/moodle///moodle.sql_.gz

[Gatt]

Ok got rsync working and it now copies our Web Server files to another host every hour

 

Have been looking at MySQL Replication - which seems to work but its locked me out of the MySQL database on the mirror server and I cant get back in

 

Update:

 

Ok disabled replication and the DB is backup now.. gonna resort to using mysqldump as above

Just need to figure out how to import the dumped files into the mirror MySQL Server..

 

 

Daily Cron Jobs:

 

Dump MySQL Database (WORKS!)

Rsync Website & MySQL Dumps to Mirror Server (WORKS!)

Import MySQL Dumps into Mirror Database (NEED HELP!)

 

ANy ideas how to run the 1st cron job in reverse?

Edited February 12, 2009 by Gatt

[CyberNerd]

 

 

Daily Cron Jobs:

 

Dump MySQL Database (WORKS!)

Rsync Website & MySQL Dumps to Mirror Server (WORKS!)

Import MySQL Dumps into Mirror Database (NEED HELP!)

 

 

You could directly import the database from a live server, without going through dumpfiles.

 

mysqldump -u 'username' -p  --opt db_name | mysql -u 'username'  -p  -h 'xxx.xxx.xxx.xxx' db_name

 

or to import a mysql dump from a file

 

mysql -u 'username'  -p  -h 'xxx.xxx.xxx.xxx' db_name < path/to/filename.sql

 

where db_name is the name of the database

[Gatt]

Thaks CyberNerd - will give that a try later

[BishopVLE]

Any tips for rsync.

I have 3 scripts that I did and am happy with as they work.

Make backup of moodledata

#/bin/sh
cp -R /srv/www/moodledata /backups/moodledatacopied/
tar -czvf /backups/moodledata/moodledata.tar.gz /backups/moodledatacopied/
mkdir -p /backups/moodledata/moodledata/`/bin/date +%Y/%b`
/bin/mv /backups/moodledata/moodledata.tar.gz /backups/moodledata/moodledata/`/bin/date +%Y/%b/moodledata_%d-%b-%Y.tar.gz`
rm -rvf /backups/moodledatacopied/

 

Make backup of moodlesoftwere

 

#/bin/sh
mkdir -p /backups/moodlesoftwerecopied/
cp -R /srv/www/htdocs/moodle /backups/moodlesoftwerecopied/moodle/
mkdir -p /backups/moodlesoftwere/
tar -czvf /backups/moodlesoftwere/moodle.tar.gz /backups/moodlesoftwerecopied/moodle/
mkdir -p /backups/moodlesoftwere/moodlesoftwere/`/bin/date +%Y/%b`
/bin/mv /backups/moodlesoftwere/moodle.tar.gz /backups/moodlesoftwere/moodlesoftwere/`/bin/date +%Y/%b/moodle_%d-%b-%Y.tar.gz`
rm -rvf /backups/moodlesoftwerecopied/

and make backup of moodle database.

 

#/bin/sh
/usr/bin/mysqldump -uroot -p****** --opt moodle > /backups/mysql_backups/moodle/moodle.sql
/usr/bin/gzip /backups/mysql_backups/moodle/moodle.sql
mkdir -p /backups/mysql_backups/moodle/`/bin/date +%Y/%b`
/bin/mv /backups/mysql_backups/moodle/moodle.sql.gz /backups/mysql_backups/moodle/`/bin/date +%Y/%b/moodle.sql_%d-%b-%Y.gz`

 

These all work, and create the files in a back up directory. But once the site grows it will be to big to do full back ups daily.

 

The SQL dump will have to be done daily as normal.

The moodlesoftwere how much will that change? would that be ok to do weekly?

But whats the best way to do the moodledata one.

I have heard about using rsync to do incremental backups but have no idea how to do that with my script.

 

TIA

[Gatt]

@BishopVLE - I'll post my scripts and references once i get a working mirror - its just teh mysqldump that needs looking into

 

THis site might help you with rsync via SSH Mirror Your Web Site With rsync | HowtoForge - Linux Howtos and Tutorials

Its for Fedora but should work with most Distros

[Gatt]

Ok got the mysqldump to work over SSH

 

mysqldump -uUSERNAME -pPASSWORD --all-databases | ssh user@mirror-server "mysql -uUSERNAME -pPASSWORD"

 

The problem is it keeps prompting for my ssh password

[powdarrmonkey]

Gatt: generate an ssh key and tell your server about it:

 

HOWTO: set up ssh keys

[CyberNerd]

The problem is it keeps prompting for my ssh password

 

passwordless ssh

How-To: "Password-less" SSH

.:: linux howtos ~ passwordless ssh ::.

Shortest passwordless ssh tutorial, ever

[Gatt]

Thanks - think its working now - just waiting on the cronjob to run ...

[Gatt]

yay it works

now have a fully working mirror server

[Gatt]

just as well it works! dropped the Moodle database from the LIVE server instead of the mirror

 

[jeverington]

If i remember at the college, i setup a NAS as a auto mount disk and used the onboard moodle backup script.

 

Ubuntu tut for setting up a auto mount

Automatically Mounting Windows SMB Shares in Ubuntu v3 | Matt Van Stone