Loading ...
Rotating Linux Log Files - Part 1: syslog
Syslog is the default logging application installed in most Linux distributions. It can be replaced with syslog-ng for better functionality, but about this in a future article. As I explained in the introduction, the log files that are managed by syslog are not rotated with logrotate, but by syslog itself. In the second part I will cover the log files that are handled by logrotate.
What files are handled by syslog? We can find out what are those files simply by inspecting the syslog configuration file (/etc/syslog.conf) that defines each log file, and also what kind of information is saved to each particular file. Let’s see how the configuration file looks on a fresh Debian system (I have removed most of the comments and kept only the relevant log definitions):
# /etc/syslog.conf Configuration file for syslogd.
#
# For more information see syslog.conf(5)
# manpage.
auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none -/var/log/syslog
#cron.* /var/log/cron.log
daemon.* -/var/log/daemon.log
kern.* -/var/log/kern.log
lpr.* -/var/log/lpr.log
mail.* -/var/log/mail.log
user.* -/var/log/user.log
uucp.* /var/log/uucp.log
...
mail.info -/var/log/mail.info
mail.warn -/var/log/mail.warn
mail.err /var/log/mail.err
news.crit /var/log/news/news.crit
news.err /var/log/news/news.err
news.notice -/var/log/news/news.notice
*.=debug;
auth,authpriv.none;
news.none;mail.none -/var/log/debug
*.=info;*.=notice;*.=warn;
auth,authpriv.none;
cron,daemon.none;
mail,news.none -/var/log/messages
Normally, I would change this and configure based on my preferences, but his is not the point now. As you can see there are various log files that will contain the information specified by the configured facility (authpriv, cron, daemon, ftp, kern, lpr, mail, mark, news, security, syslog, user, uucp and local0 through local7).
How are these files rotated? As I previously said this is handled by syslog itself, and it is done using 2 cron scripts: daily and weekly.
daily rotation: (handled by /etc/cron.daily/sysklogd)
- any files that contains the *.* facility in the syslog configuration are rotated daily. The reason for this, is that they will log all the information regardless of the facility, and can become quite quickly very big.
- if we will look inside the daily syslog cron we will see that it finds the logs it need to rotate by launching the file syslogd-listfiles:
/usr/sbin/syslogd-listfiles /var/log/syslog <- the result on my default system
- the actual rotation is handled by the savelog program as it can be seen in this line:
savelog -g adm -m 640 -u root -c 7 $LOG >/dev/null
So we can see here that by default my debian system will keep 7 archives of previous logs (7 days). If I would want to change this, then all I have to do is to replace the -c 7 parameter with what I need.
When does this rotation occur? Since it is launched from /etc/cron.daily/ it is defined in the system wide crontab:
# /etc/crontab: system-wide crontabso based on the default cron job, this will be done daily at 6:25AM.
...
25 6 * * * root test -x /usr/sbin/anacron || run-parts --report /etc/cron.daily
47 6 * * 7 root test -x /usr/sbin/anacron || run-parts --report /etc/cron.weekly
weekly rotation: (handled by /etc/cron.weekly/sysklogd)
- the rest of the syslog generated log files (different from *.* facility) will be rotated weekly.
- if we will look inside the weekly syslog cron we will see it finds the logs it needs to rotate by running:
/usr/sbin/syslogd-listfiles --weekly /var/log/mail.warn /var/log/uucp.log /var/log/user.log /var/log/daemon.log /var/log/messages /var/log/debug /var/log/auth.log /var/log/mail.err /var/log/mail.log /var/log/kern.log /var/log/lpr.log /var/log/mail.info
- as we can see all the logs defined in the syslog configuration file will appear, except the news.*, that can be included by adding syslogd-listfiles –news, if needed.
- the rotation is again handled by the savelog program:
savelog -g adm -m 640 -u root -c 4 $LOG >/dev/null
So by default it will keep 4 archives of old logs (without counting the current log); the archives will have the extension: *.0-*.3 (with the first archive not compressed by default). If I would want to change this, I would need to modify accordingly the -c 4 parameter based on my needs.
As seen above in the system crontab, this rotation will take place at 6:47AM each Sunday (the weekly cronjob).
For example, the rotated log files for the messages log file, will look like this:
/var/log/messages
/var/log/messages.0
/var/log/messages.1.gz
/var/log/messages.2.gz
/var/log/messages.3.gz Note: On RedHat based systems (RHEL, Centos, Fedora, etc.) the functionality covered above doesn’t exist by default (even though I don’t see why it could not be implemented if someone wants it). On these operating systems, this is handled also by logrotate as shown in the next part.
This covers the basics on how system logs are rotated. In part 2 we will be looking at how application logs are rotated.
Go to:
Rotating Linux Log Files - Part 2: logrotate
Rotating Linux Log Files - Introduction
Tags: Linux, logrotate, log_rotation, syslog
Share This
12th October 2006, 13:15
Hi
There is a number of files in /var/log and its sub directories that
are indeed rotated and I wonder how. These files include:
dmesg, evms-engine.log, faillog,lastlog,udev,Xorg.*.log.
I cannot find them either in /etc/logrotate.conf + includes or
among those found if you run syslogd-listfiles.
My problem is that I would like to let logrotate run all my log
rotations and must be able to stop all the other methods.
I run Ubuntu Linux 6.06 (Dapper Drake) and would be very
grateful for a tip.
11th March 2007, 02:43
hello
ther are a no of log files generated by the system itself and applications log too in linux. and they are rotated dialy , hourly ,weekly and so on. but i want to create a program that rotates all log files by the same program and other different utilities to stop log rotation.
please give me the suggestions and the help such that i can be able to do it.
i want to use the bash scripting.