Backup your MySQL databases automatically with AutoMySQLBackup
If you site relies on MySQL and stores its sensitive data in a MySQL database, you will most definitely want to backup that information so that it can be restored in case of any disaster (manual mistake to delete some data, software errors, hardware errors, server compromise, etc.). In a previous post I have presented that MySQL provides the basic tool (mysqldump) to perform database backups. This is required because backing up a database is a little different than backing up regular files.
With mysqldump anyone can write a small shell script and running it from cron, it will achieve an automatic backup solution. There are many such scripts already available freely and also many commercial solutions also (I assume as I have not tested any really ;)). The script that I liked the most is AutoMySQLBackup, because it doesn’t have any real requirements (mysqldump of course is needed - in any mysql client package - and gzip or bzip2 to compress the resulting file) and has all the features I was looking for in such a script.
AutoMySQLBackup has all the features I needed: it can backup a single database, multiple databases, or all the databases on the server; each database is saved in a separate file that can be compressed (with gzip or bzip2); it will rotate the backups and not keep them filling your hard drive (as normal in the daily backup you will have only the last 7 days of backups, the weekly if enabled will have one for each week, etc.). It has also some other features (check the project homepage for full details), that I am not using myself (like email logs for example), but other peoples might find interesting.
The installation is very simple: just download the one file bash script and save it somewhere, customize it to fit your setup (only some basic changes are needed: like the MySQL user and password, backup location), make it executable and activate it in cron as needed (daily for example).
Here are the variables that I usually setup:
# Username to access the MySQL server e.g. dbuser
USERNAME=dbuser
# Username to access the MySQL server e.g. password
PASSWORD=password
# Host name (or IP address) of MySQL server e.g localhost
DBHOST=localhost
# List of DBNAMES for Daily/Weekly Backup e.g. "DB1 DB2 DB3"
DBNAMES="all"
# Backup directory location e.g /backups
BACKUPDIR="/var/backup/mysql"
# Mail setup
MAILCONTENT="quiet"
You can also run the script manually:
./automysqlbackup.sh.2.5
but probably you will want to enable it in cron and run daily. You can just dump it in the /etc/cron.daily/ folder and you should be done. :).
Myself I like to place it in /opt and create a symlink to the file (to help with future updates of the script):
ln -s automysqlbackup.sh.2.5 automysqlbackup.sh
and run it from cron by placing in /etc/crontab something like:
#MySQL Daily backup
45 5 * * * root /opt/automysqlbackup.sh >/dev/null 2>&1
If needed, after the script is finished, you can save the backup folder where you placed the databases to a remote location (ftp, nfs, smb, or whatever) or another local medium (like tape for ex.). Saving to remote locations is not supported in the script itself so you will have to either mount the remote space locally (if it is possible… for nfs, smb, for ex.) and save directly onto it, or you will have to use another method to upload the files remotely (maybe you just have to include the backup folder used above in your regular backup script).
Are you using a different scrip to backup your databases? I would like to hear of it… Why did you like it? What features were you looking for? Or you had to write your own script because you have not found your needed features in any existing script available freely? Let me know…
Share This







28th May 2006, 13:15
Good help!
However, it seems to me that if users are unable to access the server console, then there is no way to backup the MySQL directly over the website, or http://FTP…etc.
28th May 2006, 13:44
Keith,
Indeed this script is intended to be launched from the console (from cron in general).
Still:
- in case you can access the MySQL server remotely (if it is listening on a public IP and you have a user and password with proper privileges from another location), you can still use this script from another server.
- in case you have access to run your own cron jobs (from a control panel software for example) and you can upload files to the server, you can still launch the script like that (just place it somewhere on the server and with the proper user/pass you can still use it, by enabling it in your own crontab).
28th May 2006, 14:46
wow, this is exactly what I need. Installed and works a treat. You mentioned you can set it up to backup to a remote location. I couldn’t see this in any of the documentation? I would like it to ftp the backups to a remote server somehow.. got any clues?
28th May 2006, 14:59
Marc,
I may have expressed myself not very clear… Sorry about that. You can save with this script only to a local storage. You can mount a remote storage (nfs, smb, for ex.) and save like that on a remote location. If not, you will have to use a different approach to save the local backup folder to a remote location (just as you might be doing for your regular files).
28th May 2006, 19:56
check back-up buddy:
* Set what time, how often and where to store your MySQL files on your local machine.
* One click backup now feature allows you to run your backups right now.
* Minimizes down to your Windows Task Tray for easy access.
http://www.successfulhosting.com/features/buddy.php
28th May 2006, 20:13
Seems a little too complicated. A simple one-line cron job is all you need:
30 2 * * * mysqldump -udebian-sys-maint –password=xxx -hlocalhost –all-databases –opt –allow-keywords –flush-logs –hex-blob –master-data –max_allowed_packet=16M –quote-names –result-file=/home/user/backup/mysql_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.sql
29th May 2006, 07:50
an alternative implementation of a similar way to backup mysql is mysqlblasy: http://freshmeat.net/projects/mysqlblasy/
cheers
29th May 2006, 14:04
I’d like to add to comment #6 that this “script” works easily on remote backup servers in conjunction with SSH. So instead of cron’ing the mysqldump on the database machine, I’d suggest croning a “ssh mysqlservername mysqldump …” on the remote (backup storage) machine. I’d suggest setting up a no-privileges account for this and use password-less certificate authentication for SSH.
30th May 2006, 08:40
Disclaimer: I haven’t tested this script…
My experience with mysqldump is rather disappointing.
The database I’m currently running for a few sites is about 20GB in size (several tables with 1-15 million records), and averages around 60 qps.
mysqldump doesn’t even come close to working even slightly “ok” for such a database.
I would be very interested in suggestions for how to do easy backups of such a database. Currently I’m using a replication server, which again has hot swappable disks on a rotation. But MySQL replication isn’t exactly 100% perfect either.
Just looking for some pointers to more information/tips of how to run such a database
30th May 2006, 08:51
Erlend,
Well, in this case any script that is based on mysqldump (as this one) will not help you very much… There are some tricks that can improve mysqldump performance, but this will not help you much in your case.
What kind of tables are you using? MyISAM or InnoDB?
Have you experimented with mysqlhotcopy? You might want to look into it, and see how it works on your setup.
30th May 2006, 12:51
Hi
Thanks for the tip!
Just tried out mysqlhotcopy now. It took 1505 seconds to do a copy of all tables and databases without the index-files. Which is pretty decent. It has some more features I’m going to check into now.
It’s a bit annoying that it does a lock on all databases and tables. I suppose I could always rund a shellscript which would loop through a list of all databases, copying one at a time.
30th May 2006, 13:53
Erlend,
This is why I have asked you about InnoDB… In that case you cold have used mysqldump with the –single-transaction parameter.
Anyway, I am happy that I could help you. Good luck, and let me know how it went.
31st May 2006, 10:06
Webmin has a nice backup feature in its MySQL module that I’ve used over the past year. Getting backups going took about 3 clicks.
31st May 2006, 10:18
Pete,
Yep… You can even do that very easy from PhpMyAdmin. But that will not be so easy to automate, right? Anyway thanks for your message… there are many other solutions available to do this; I just presented a very simple and effective one that I have liked.
Cheers!
31st May 2006, 15:20
Here’s a tip… I have my dumps stored in RCS. After each dump is downloaded, it gets checked into RCS (Revision Control System) so that only the delta is stored. Saves a ton of space on backups, and I can check out any previous revision that I haven’t purged. Check it out. If you are keeping 7 days of backups, 6 of them will be stored as merely a sort of incrimental.
*Darb
1st July 2006, 03:35
Good site! Good information. Keep Alive!
21st July 2006, 23:19
[...] Backup your MySQL databases automatically with AutoMySQLBackup | MDLog:/sysadmin (tags: mysql) [...]
1st September 2006, 12:10
[...] Automatically backup your MySQL databases with this simple but very effective script…read more | digg story [...]
2nd October 2006, 07:58
[...] Backup your MySQL databases automatically with AutoMySQLBackup. [...]
1st December 2006, 12:37
Hey Guys..
I need some tips regarding incremental backup.is there anybody to help me up.
Regards
bhamboure
14th December 2006, 09:20
bhamboure,
This is probably not the best place for something like that (and you would better use forums, etc), but sure, tell me what you need and if I will have some available time I will help you out.
17th December 2006, 03:25
links for 2006-12-17…
Automatic MySQL Backup
(tags: mysql backup scripts)
Backup your MySQL databases automatically with AutoMySQLBackup | MDLog:/sysadmin
(tags: mysql backup scripts)
…
25th January 2007, 05:20
[...] Read more at MDLog:/sysadmin [...]
24th February 2007, 10:40
Take a look at mysqlhotcopy. You can write a simple cron job to mysqlhotcopy an entire database. My 3 million rows get copied in 3 seconds. I zip them in the next minute in cron through a tar command. Why does one need scripts.
1st June 2007, 21:42
Hi
I use cPanel and I dont know how should I use Cron Jobs for running this script .
Please guide me .
1st June 2007, 21:49
Omid,
If you are using CPanel you should be better using the built-in backup feature of CPanel. That can save for you besides the system files also your mysql databases.
3rd September 2007, 14:40
Hi,
Can you tell me how to restore it again into database.
3rd September 2007, 15:58
Laj: you just locate the file (db) you want to restore, uncompress it and restore it with mysqldump:
4th January 2008, 21:45
Is there a way to use this to do replication???
- Shelon Padmore
4th January 2008, 21:48
Shelon: this has nothing to do with replication. Still if you have a master/slave mysql setup you will normally run your backups (this script or another one) on the slaves and not on the master. Hth.
- Marius -
12th April 2008, 20:08
I am wanting to diagnose and repair the following errors:
line 644: unexpected EOF while looking for matching “’
line 689: syntax error: unexpected end of file
Thank you
2nd May 2008, 15:53
[...] MDLog:/sysadmin - Backup your MySQL databases automatically with AutoMySQLBackup [...]
10th June 2008, 19:50
Joe: unfortunately that information only is not enough to help you troubleshoot that error. Please try to send more infos, like what os you are running, what mysql server, and if this happens when you backup a particular database or not. -M
26th June 2008, 20:22
Awesome! Exactly what I was looking for!
G-Man
27th June 2008, 08:48
Marius: OS is Linux; MySQL server is 5.0. I have only ever backed up one database; it was working fine for a long time, until I started getting these errors posted daily to log files. The process still works; it’s just these logs files that are a hassle, as I have to keep purging them.
At http://pastebin.com/m65fea9eb you can see the changes that I have made to the script; sensitive values omitted.
Please let me know what else I can provide to help diagnose this issue; I appreciate your assistance.
27th June 2008, 15:44
Joe: make sure you have in the DBNAMES parameter the name of the db you want to backup. Also try to set: MAILCONTENT=”quiet”.
Also see the bug reports already open: http://sourceforge.net/tracker/?atid=628964&group_id=101066&func=browse and try to explain there your problem also as the author is the best person to help you out on this.
hth. -M