Loading ...
Rotating Linux Log Files - Part 2: logrotate
logrotate is the default application used to rotate all other log files not handled by syslog itself (details on rotating system log files can be found in part 1 of the article). It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.
Normally, logrotate is run as a daily cron job. Let’s look into the script that was installed in /etc/cron.daily for this:
cat /etc/cron.daily/logrotate
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.confLogrotate will look into /etc/logrotate.conf for its configuration directives.
cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
missingok
monthly
create 0664 root utmp
rotate 1
}
/var/log/btmp {
missingok
monthly
create 0664 root utmp
rotate 1
}
# system-specific logs may be configured hereSo we can see it defines some default parameters (weekly, rotate 4, create, compress) and includes all the files from /etc/logrotate.d/. Also it defines the rotation for some files that are not handled by syslog itself, like wtmp. For example, I would want to keep more than one month of old wtmp logs, then I would have to change the parameter rotate 1.
Inside the /etc/logrotate.d/ various packages will install their own configuration file that will ensure their logs are properly rotate (on my fresh Debian install I have the following files: acpid apache2 aptitude base-config dpkg exim4-base). As long as you don’t change the paths to those logs the rotation will work out of the box. But in case you change them you might want to look inside this folder and make the proper adjustments to the log file definitions, to assure they will be rotated. For example, let’s look at the apache rotation file created here by the apache2 package:
cat /etc/logrotate.d/apache2
/var/log/apache2/*.log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if [ -f /var/run/apache2.pid ]; then
/etc/init.d/apache2 restart > /dev/null
fi
endscript
}We can see that by default it will rotate apache logs found in /var/log/apache2/ that have the extension *.log, on a weekly basis and keep 52 archives (about 1 year) of the old data. Once the rotation is completed it will restart the apache daemon. You can check logrotate manual page for all the available parameters, as they are self-explanatory.
Now, if I would like to keep my own apache log files in a different location (/var/weblogs for example) and rotate them monthly then I will need to make the following changes:
/var/weblogs/*.log {
monthly
...Probably, I will also want to change the default hour when the daily cron is running to have it on midnight. Anyway this is just an example and you will most certainly configure this based on your needs.
Even though I didn’t intended with this article to describe what each configuration parameter of logrotate means (as you can easily find out yourself), but to show what is the logic and its functionality, I would like to add that while configuring and testing this you might find very useful the debug option:
logrotate -d fileThis will show you what it will do, without actually rotating anything, and this is most valuable while testing complex setups that you don’t want to ‘play’ with the logs to see if your configuration will work as you want it.
Also logrotate -f file will force the rotation even if that would have normally not occurred (logrotate will only assume it need to run and rotate logs once per day).
Note: as mentioned also in part 1, RedHat based systems (RHEL, Centos, Fedora, etc.) will also rotate by default the ’system logs’ using logrotate and not syslog’s internal method as Debian systems. This is handled by default with the logrotate configuration file:
cat /etc/logrotate.d/syslog
/var/log/messages /var/log/secure /var/log/maillog /var/log/spooler /var/log/boot.log /var/log/cron {
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
endscript
}
The sharedscripts parameter means that the postrotate script will only be run once (after the old logs have been compressed), not once for each log which is rotated.
So nothing special defined here, besides the log files that will be rotated, and it will use the defaults from /etc/logrotate.conf.
Go to:
Rotating Linux Log Files - Introduction
Rotating Linux Log Files - Part 1: syslog
Tags: Linux, logrotate, log_rotation, syslog
8th June 2006, 17:42
Very Very Nice Article related to Rotating files in GNU Linux.
15th July 2006, 19:28
Took out the mystery of the auth.log rotation in my debian server. I was trying to find it in logrotate.d
Thanks!
9th October 2006, 12:48
Really good
2nd March 2007, 13:58
very intresting article
31st July 2007, 12:46
Great article, thanks.
I´d like to ask you, if you know of a method to force active processes to start writing to the empty log without restarting the process. Reason is, I´m admin of a high volume app, which needs log rotation every couple days, I have set up logrotate to do so when the log is more than 500mb, even tough the rotation succeeds, the app continues writing to the rotated log (applog.log.1) and would not start writing right away to the new one (applog.log).
Restarting the aplication is not desirable, since it would take a couple hours, therefore I´d like to know if there is a way to force an active app to start writing to the new log.
Thanks!
Loco
11th September 2007, 18:50
Like the happy fellow above, I too was seeking the mystery of auth.log rotation on Ubuntu. Thanks!
4th January 2008, 02:26
To automatically write to the new log file and not the just renamed one you need to do a `kill -HUP’ in the postrotate section. Of course you need to catch this signal in your application as well. Just catch it, fclose and fopen your log file again, and the new writes will go to the new file descriptor which will go to the just created file. Be sure to open in “a” mode too.
24th April 2008, 18:59
Apache works very well with the cronolog utility, as long as you are logging all vhosts to a single access logfile. It is called from the Apache config, and it looks after closing the old file and opening the new, based on the date and time format as specified in the Apache config file.
It does not do compression or moving files away from /var/log, but a custom script (called from cron) can do that.
It may or may not work with syslog, but a lot of customisation would be required to disable the existing rotation.
10th June 2008, 19:58
Dec: thank you for your comment. This post was not about apache, and apache was given just as an example, but I agree cronolog is great tool and can be useful in some situations. Btw. apache has built-in a similar (though lacking some of the cronolog features) called rotatelogs. -M
4th March 2009, 04:18
[...] not sure if I have this last piece correct or not, but based on what I read in this “Rotating Linux Log Files – Part 2” article I think I’ve got it right. The concern is that we don’t want the new watchguard.log [...]