Whatever web log analyzer you would use, at one time or the other you will end up with one problem: how to deal with weblogs rotation, to not have a gap in your statistics? This has various solutions starting with running the weblog analyzer at the prerotate step of logrotate, or maybe using some manual script.
This post will show how you can rotate the apache logs using awstats right after it has processed the logs. This can be beneficial for situations where you have quite big logs and using this method will keep them small all the time, and also where restarting apache just for logrotating is not such a good idea. Obviously for this to make sense, you need to be already using awstats for your log processing .

Read the rest of this entry »

logrotate is the default application used to rotate all other log files not handled by syslog itself (details on rotating system log files can be found in part 1 of the article). It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large. Read the rest of this entry »

Syslog is the default logging application installed in most Linux distributions. It can be replaced with syslog-ng for better functionality, but about this in a future article. As I explained in the introduction, the log files that are managed by syslog are not rotated with logrotate, but by syslog itself. In the second part I will cover the log files that are handled by logrotate.

Read the rest of this entry »

Logs… Any Linux system will generate many log files by default, containing various information about the operation of the system (normal actions, debugging information, security/authorization messages, web/email events, etc). If no rotation would occur on the various log files, then they will just grow bigger and bigger, filling up the space (on high traffic sites) but most importantly making it very difficult to find any information that we might be looking for in those log files. Fortunately this is handled in most Linux distributions by default and we don’t have anything special to do to set it up… if will function out of the box, rotating the log files by default.

This little how-to will show you how the default log rotation works, based on syslog and logrotate. There are other ways to achieve this, like using syslog-ng instead of syslog that I will cover in a future article.
Understanding how the default setup works, will help you have an idea of what will happen and what logs will be rotated, when will this occur, and how long will they be kept. Also it will show you the places where you can make changes in case you need to do that (if you want to save some log for a longer time, or if you want to rotate it differently from the default). The examples I will present are taken from a Debian system, so if you are running a different system, they might differ a little, but not drastically (like you might have the cron setup to run at a different time, or keep a different number of log iterations by default).

Read the rest of this entry »