Backup your MySQL databases automatically with AutoMySQLBackup
If you site relies on MySQL and stores its sensitive data in a MySQL database, you will most definitely want to backup that information so that it can be restored in case of any disaster (manual mistake to delete some data, software errors, hardware errors, server compromise, etc.). In a previous post I have presented that MySQL provides the basic tool (mysqldump) to perform database backups. This is required because backing up a database is a little different than backing up regular files.
With mysqldump anyone can write a small shell script and running it from cron, it will achieve an automatic backup solution. There are many such scripts already available freely and also many commercial solutions also (I assume as I have not tested any really ;)). The script that I liked the most is AutoMySQLBackup, because it doesn’t have any real requirements (mysqldump of course is needed - in any mysql client package - and gzip or bzip2 to compress the resulting file) and has all the features I was looking for in such a script.
AutoMySQLBackup has all the features I needed: it can backup a single database, multiple databases, or all the databases on the server; each database is saved in a separate file that can be compressed (with gzip or bzip2); it will rotate the backups and not keep them filling your hard drive (as normal in the daily backup you will have only the last 7 days of backups, the weekly if enabled will have one for each week, etc.). It has also some other features (check the project homepage for full details), that I am not using myself (like email logs for example), but other peoples might find interesting.
The installation is very simple: just download the one file bash script and save it somewhere, customize it to fit your setup (only some basic changes are needed: like the MySQL user and password, backup location), make it executable and activate it in cron as needed (daily for example).
Here are the variables that I usually setup:
# Username to access the MySQL server e.g. dbuser
USERNAME=dbuser
# Username to access the MySQL server e.g. password
PASSWORD=password
# Host name (or IP address) of MySQL server e.g localhost
DBHOST=localhost
# List of DBNAMES for Daily/Weekly Backup e.g. "DB1 DB2 DB3"
DBNAMES="all"
# Backup directory location e.g /backups
BACKUPDIR="/var/backup/mysql"
# Mail setup
MAILCONTENT="quiet"
You can also run the script manually:
./automysqlbackup.sh.2.5
but probably you will want to enable it in cron and run daily. You can just dump it in the /etc/cron.daily/ folder and you should be done. :).
Myself I like to place it in /opt and create a symlink to the file (to help with future updates of the script):
ln -s automysqlbackup.sh.2.5 automysqlbackup.sh
and run it from cron by placing in /etc/crontab something like:
#MySQL Daily backup
45 5 * * * root /opt/automysqlbackup.sh >/dev/null 2>&1
If needed, after the script is finished, you can save the backup folder where you placed the databases to a remote location (ftp, nfs, smb, or whatever) or another local medium (like tape for ex.). Saving to remote locations is not supported in the script itself so you will have to either mount the remote space locally (if it is possible… for nfs, smb, for ex.) and save directly onto it, or you will have to use another method to upload the files remotely (maybe you just have to include the backup folder used above in your regular backup script).
Are you using a different scrip to backup your databases? I would like to hear of it… Why did you like it? What features were you looking for? Or you had to write your own script because you have not found your needed features in any existing script available freely? Let me know…
Brought to you by: online backup services
>
Tags: backup, mysql, mysqldump







28th May 2006, 13:15
Good help!
However, it seems to me that if users are unable to access the server console, then there is no way to backup the MySQL directly over the website, or http://FTP…etc.
28th May 2006, 13:44
Keith,
Indeed this script is intended to be launched from the console (from cron in general).
Still:
- in case you can access the MySQL server remotely (if it is listening on a public IP and you have a user and password with proper privileges from another location), you can still use this script from another server.
- in case you have access to run your own cron jobs (from a control panel software for example) and you can upload files to the server, you can still launch the script like that (just place it somewhere on the server and with the proper user/pass you can still use it, by enabling it in your own crontab).
28th May 2006, 14:46
wow, this is exactly what I need. Installed and works a treat. You mentioned you can set it up to backup to a remote location. I couldn’t see this in any of the documentation? I would like it to ftp the backups to a remote server somehow.. got any clues?
28th May 2006, 14:59
Marc,
I may have expressed myself not very clear… Sorry about that. You can save with this script only to a local storage. You can mount a remote storage (nfs, smb, for ex.) and save like that on a remote location. If not, you will have to use a different approach to save the local backup folder to a remote location (just as you might be doing for your regular files).
28th May 2006, 19:56
check back-up buddy:
* Set what time, how often and where to store your MySQL files on your local machine.
* One click backup now feature allows you to run your backups right now.
* Minimizes down to your Windows Task Tray for easy access.
http://www.successfulhosting.com/features/buddy.php
28th May 2006, 20:13
Seems a little too complicated. A simple one-line cron job is all you need:
30 2 * * * mysqldump -udebian-sys-maint –password=xxx -hlocalhost –all-databases –opt –allow-keywords –flush-logs –hex-blob –master-data –max_allowed_packet=16M –quote-names –result-file=/home/user/backup/mysql_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.sql
29th May 2006, 07:50
an alternative implementation of a similar way to backup mysql is mysqlblasy: http://freshmeat.net/projects/mysqlblasy/
cheers
29th May 2006, 14:04
I’d like to add to comment #6 that this “script” works easily on remote backup servers in conjunction with SSH. So instead of cron’ing the mysqldump on the database machine, I’d suggest croning a “ssh mysqlservername mysqldump …” on the remote (backup storage) machine. I’d suggest setting up a no-privileges account for this and use password-less certificate authentication for SSH.
30th May 2006, 08:40
Disclaimer: I haven’t tested this script…
My experience with mysqldump is rather disappointing.
The database I’m currently running for a few sites is about 20GB in size (several tables with 1-15 million records), and averages around 60 qps.
mysqldump doesn’t even come close to working even slightly “ok” for such a database.
I would be very interested in suggestions for how to do easy backups of such a database. Currently I’m using a replication server, which again has hot swappable disks on a rotation. But MySQL replication isn’t exactly 100% perfect either.
Just looking for some pointers to more information/tips of how to run such a database
30th May 2006, 08:51
Erlend,
Well, in this case any script that is based on mysqldump (as this one) will not help you very much… There are some tricks that can improve mysqldump performance, but this will not help you much in your case.
What kind of tables are you using? MyISAM or InnoDB?
Have you experimented with mysqlhotcopy? You might want to look into it, and see how it works on your setup.
30th May 2006, 12:51
Hi
Thanks for the tip!
Just tried out mysqlhotcopy now. It took 1505 seconds to do a copy of all tables and databases without the index-files. Which is pretty decent. It has some more features I’m going to check into now.
It’s a bit annoying that it does a lock on all databases and tables. I suppose I could always rund a shellscript which would loop through a list of all databases, copying one at a time.
30th May 2006, 13:53
Erlend,
This is why I have asked you about InnoDB… In that case you cold have used mysqldump with the –single-transaction parameter.
Anyway, I am happy that I could help you. Good luck, and let me know how it went.
31st May 2006, 10:06
Webmin has a nice backup feature in its MySQL module that I’ve used over the past year. Getting backups going took about 3 clicks.
31st May 2006, 10:18
Pete,
Yep… You can even do that very easy from PhpMyAdmin. But that will not be so easy to automate, right? Anyway thanks for your message… there are many other solutions available to do this; I just presented a very simple and effective one that I have liked.
Cheers!
31st May 2006, 15:20
Here’s a tip… I have my dumps stored in RCS. After each dump is downloaded, it gets checked into RCS (Revision Control System) so that only the delta is stored. Saves a ton of space on backups, and I can check out any previous revision that I haven’t purged. Check it out. If you are keeping 7 days of backups, 6 of them will be stored as merely a sort of incrimental.
*Darb
1st July 2006, 03:35
Good site! Good information. Keep Alive!
21st July 2006, 23:19
[...] Backup your MySQL databases automatically with AutoMySQLBackup | MDLog:/sysadmin (tags: mysql) [...]
1st September 2006, 12:10
[...] Automatically backup your MySQL databases with this simple but very effective script…read more | digg story [...]
2nd October 2006, 07:58
[...] Backup your MySQL databases automatically with AutoMySQLBackup. [...]
1st December 2006, 12:37
Hey Guys..
I need some tips regarding incremental backup.is there anybody to help me up.
Regards
bhamboure
14th December 2006, 09:20
bhamboure,
This is probably not the best place for something like that (and you would better use forums, etc), but sure, tell me what you need and if I will have some available time I will help you out.
17th December 2006, 03:25
links for 2006-12-17…
Automatic MySQL Backup
(tags: mysql backup scripts)
Backup your MySQL databases automatically with AutoMySQLBackup | MDLog:/sysadmin
(tags: mysql backup scripts)
…
25th January 2007, 05:20
[...] Read more at MDLog:/sysadmin [...]
24th February 2007, 10:40
Take a look at mysqlhotcopy. You can write a simple cron job to mysqlhotcopy an entire database. My 3 million rows get copied in 3 seconds. I zip them in the next minute in cron through a tar command. Why does one need scripts.
1st June 2007, 21:42
Hi
I use cPanel and I dont know how should I use Cron Jobs for running this script .
Please guide me .
1st June 2007, 21:49
Omid,
If you are using CPanel you should be better using the built-in backup feature of CPanel. That can save for you besides the system files also your mysql databases.
3rd September 2007, 14:40
Hi,
Can you tell me how to restore it again into database.
3rd September 2007, 15:58
Laj: you just locate the file (db) you want to restore, uncompress it and restore it with mysqldump:
4th January 2008, 21:45
Is there a way to use this to do replication???
- Shelon Padmore
4th January 2008, 21:48
Shelon: this has nothing to do with replication. Still if you have a master/slave mysql setup you will normally run your backups (this script or another one) on the slaves and not on the master. Hth.
- Marius -
12th April 2008, 20:08
I am wanting to diagnose and repair the following errors:
line 644: unexpected EOF while looking for matching “’
line 689: syntax error: unexpected end of file
Thank you
2nd May 2008, 15:53
[...] MDLog:/sysadmin - Backup your MySQL databases automatically with AutoMySQLBackup [...]
10th June 2008, 19:50
Joe: unfortunately that information only is not enough to help you troubleshoot that error. Please try to send more infos, like what os you are running, what mysql server, and if this happens when you backup a particular database or not. -M
26th June 2008, 20:22
Awesome! Exactly what I was looking for!
G-Man
27th June 2008, 08:48
Marius: OS is Linux; MySQL server is 5.0. I have only ever backed up one database; it was working fine for a long time, until I started getting these errors posted daily to log files. The process still works; it’s just these logs files that are a hassle, as I have to keep purging them.
At http://pastebin.com/m65fea9eb you can see the changes that I have made to the script; sensitive values omitted.
Please let me know what else I can provide to help diagnose this issue; I appreciate your assistance.
27th June 2008, 15:44
Joe: make sure you have in the DBNAMES parameter the name of the db you want to backup. Also try to set: MAILCONTENT=”quiet”.
Also see the bug reports already open: http://sourceforge.net/tracker/?atid=628964&group_id=101066&func=browse and try to explain there your problem also as the author is the best person to help you out on this.
hth. -M
16th November 2008, 01:14
[...] Backup MySQL Databases Automatically [...]
16th January 2009, 05:41
[...] http://www.ducea.com/2006/05/27/backup-your-mysql-databases-automatically-with-automysqlbackup/ [...]
6th February 2009, 06:04
[...] Backup MySQL Databases Automatically [...]
24th February 2009, 09:26
Hello,
So, on Dreamhost, what is the best way to automatically backup database every night ?
Does using a cron job works well ?
Thanks.
24th February 2009, 09:31
@kay: hmm… not sure what are the limitations on Dreamhost. If you are on a shared system you will probably want to do this from your control panel, or using a simple php script. If you let me know what are your limitations, I might be able to give you some more ideas
24th February 2009, 09:46
Sorry but I don’t know what you mean by limitations ?
I am on private server Dreamhost PS.
Thanks for help.
24th February 2009, 09:50
@kay: in that case use the script explained in this post; it should work just fine. hth.
5th March 2009, 21:44
hi, Love your script but having a problem getting it to backup a DB on another server. I have included the IP address but it still thinks it is on the current server. The error I get is
mysqldump: Got error: 1045: Access denied for user ‘airattac_****’@'122.**.**.**’ (using password: YES) when trying to connect.
This is the current server IP where the server I want to run the backup from is a different IP.
Any ideas please?
PSD. I do not have shell access on the other server only cron.
5th March 2009, 21:47
Let me explain that a little better……. I am trying to backup a DB from another server and have included this in the script. When I run the script manually I get the error above. The server I am running the script from is the 122.**.**.** IP but the server I want to back up is a different IP and I have that configured in the script.
How do I get it to look at the correct IP address?
Thanks
Glenn
5th March 2009, 21:56
@Glenn: you only have to set the DBHOST variable for this. The script can run only one configuration, so if you need 2 (1 local and one remote) you should probably make a copy and run them like that separate.
Now, the error you are seeing is probably telling you (from what I understand from your comments) that the mysql user has no permission to login. You should be able to test this manually with:
mysql -h hostname -u user -pif that works, the script should work also. If that doesn’t work, fix that first and get back to the script after that… (probably your other db has the user permission only on the localhost, and not from the remote ip you are trying). hth.
5th March 2009, 22:34
That didnt work although I have 2 scripts. I use 1 script for an AU server where I am using this script for a US server and it seems I cant access it.
While I was writing this it just occurred to me to add the IP from the other server to the cpanel on the US server. This believe it or not actually worked.
Thanks for your prompt response…it triggered my brain to work
Glenn
5th March 2009, 22:38
Just another quick question…how can I backup every DB on the server at once…would I need to add the same user to every DB on every cpanel and then add each DB to the script or should I just run multiple scripts?
Glenn
6th March 2009, 08:41
@Glenn: using automysqlbackup you just have to set DBNAMES=”all”; though this will probably not work on your remote cpanel server where you probably don’t have access to *all* the dbs on the server. In that case just add all the dbs you need, like MDBNAMES=”db1 db2 db3″. hth.
9th March 2009, 21:37
Thanks Marius,
What I have actually done and found easier was to have individual files. This way they are all set to “all” and copy everything in each account.
Thanks for the script, it was just what I needed
Glenn
9th March 2009, 22:05
@Glenn: I am happy you were able to find a solution to your problem. About the script as I mentioned in this post it is not written by me, I just reviewed it and publicized it a little. You have the author home page on the top of the post; feel free to thank him directly if you want by sending him a paypal gift
.
Cheers,
- Marius -
10th March 2009, 01:12
[...] or there are tools you can download and run as a cron to automatically backup databases like this Backup your MySQL databases automatically with AutoMySQLBackup | MDLog:/sysadmin You will also want to backup your actual files via ftp every week or so The price should be the [...]
15th March 2009, 22:35
[...] 3. Backup your MySQL databases automatically with AutoMySQLBackup [...]
16th March 2009, 01:52
I have installed the script and it created daily, weekly, monthly folder in /backups. However the three databases that I have were backed up with no data in them. The named database appears but it is empty.
??
16th March 2009, 09:03
@greg: run it by hand and see if it gives any errors or warnings. Please ensure that your user has proper permissions (maybe run a mysqldump by hand with the same user/pass to ensure all is ok). Hth, M.
16th March 2009, 18:03
Backup is at the top of my frustrations with MySQL. Different options required for different table types => how to do it for a db with mixed table types.
A server shutdown then dump is the only reasonable solution I can find.
17th March 2009, 09:50
@John: I am not sure what you mean… mysqldump for example should work just fine with different tables types; I don’t see any problem with that and why you would need to shutdown anything for this. Maybe you are backing up raw tables, etc. but this really has nothing to do with this script, or with mysql (as any other software will have such issues). If you need any help on this feel free to contact me using the contact form on my blog.
18th March 2009, 07:48
[...] Via | Ducea [...]
19th March 2009, 22:22
I am trying to get this script working, using Cpanel and cron jobs
placed the script into /etc/cron.daily folder
set the variables in the script before uploading……
created a folder named /backups in public_html
have set the cron tabs command line
but nothing seems to get backed up
How do I call the script from my browser?
I would like to see if it runs at all……
inmobiz
20th March 2009, 07:48
@inmobiz: you can’t run it in a browser. Test it from a console to see if all is ok. If it runs fine, the cron should run ok also. hth.
20th March 2009, 07:50
[...] 3. Backup your MySQL databases automatically with AutoMySQLBackup [...]
25th March 2009, 15:57
[...] 3. Backup your MySQL databases automatically with AutoMySQLBackup [...]
31st March 2009, 23:19
[...] 3. Backup your MySQL databases automatically with AutoMySQLBackup [...]
5th April 2009, 11:06
[...] AutoMySQLBackup posee algunas características muy buenas para realizar un respaldo de una base de datos sola, de múltiples bases de datos o de todas las bases de datos del servidor. Cada base de datos se guarda en un archivo separado que podrá ser comprimido (con gzip o bzip2); rotará los respaldos y no los mantendrá ocupando espacio en tu disco. (Más info) [...]
7th May 2009, 06:19
Muchas gracias! Life seems to be better!