Many times you might want to fine tune the default permissions of the files created on a linux system. This is very simple and usually if you are using bash all you have to do is to define somewhere in the bash startup files (/etc/profile is a good place for this) a new value for umask like this:
umask 002
(this will allow by default group write permissions on the newly created files)

Normally on modern linux distributions this is by default set to 022 and you can easily find out what it is on your system by running the umask command:
umask

Contrary to what you might think, this is not enough to have this working for all applications and daemons on the system. This works fine for any files created from a shell session, but the files created by other processes, like the web server for example, will still use the default, unless otherwise configured. In order to have apache use a different umask we can define this inside /etc/apache2/envvars (debian, and ubuntu systems) or /etc/sysconfig/httpd (rhel,centos systems) like this:
umask 002
and restart apache to enable it.

Other daemons will have different locations where you can define this to overwrite the default setting for umask (check their documentation if you are unsure).

The moment a PHP application grows to run on more servers, normally people will see problems caused by PHP sessions. If the application is not persistent you are lucky and don’t care about this, but if not you will quickly see this regardless of how good the load balancer you use is handling stickiness (sending the users to the same real server), this will slowly become a major issue. There are various solutions that can be used to store PHP sessions in a shared location, but I want to present today one solution that is very simple to implement, yet very efficient and on the long term better suited than using a database backend for this: using memcache to store the sessions.

The pecl memcache php extension has supported for a long time the memcache session.save_handler, but with the release 3.0.x (still in beta at this time) this brings in a set of interesting features for us:
- UDP support
- Binary protocol support
- Non-blocking IO using select()
- Key and session redundancy (values are written to N mirrors)
- Improved error reporting and failover handling

Read the rest of this entry »

iotop does for I/O usage what top does for CPU usage. It watches I/O usage information output by the Linux kernel (requires 2.6.20 or later) and displays a table of current I/O usage by processes on the system. This tool is written by Guillaume Chazarain and requires Python >= 2.5 and a Linux kernel >= 2.6.20 to run. This post introduces this very useful tool and shows how we can install it and use it.

iotop can be downloaded either as source package or a rpm package. Starting with lenny, debian includes iotop in the main repository and it can be installed just as simple as running:
aptitude install iotopThis is very cool indeed and kudos to the debian team to include iotop in lenny
Read the rest of this entry »

We all know and love vmstat, but wouldn’t it be nice to get such information on a per process basis, to be able to better understand what is causing i/o problems? This is exactly what iopp, written by Mark Wong and released as open source does:
“It’s a custom tool to go through the Linux process table to get i/o statistics per process. It is open source and can be downloaded from: http://git.postgresql.org/?p=~markwkm/iopp.git;a=summary“

Now this sounds interesting, and I am sure anyone that has dealt with i/o issues in the past will probably find this very useful. Let’s see how we can install it and what kind of reporting we get. We will install this from source and here are some quick steps to do this (you will need git and cmake for this):
git clone git://git.postgresql.org/git/~markwkm/iopp.git
cd iopp
cmake CMakeLists.txt
make

Read the rest of this entry »

Mdadm is the modern tool most Linux distributions use these days to manage software RAID arrays; in the past raidtools was the tool we have used for this. This cheat sheet will show the most common usages of mdadm to manage software raid arrays; it assumes you have a good understanding of software RAID and Linux in general, and it will just explain the commands line usage of mdadm. The examples bellow use RAID1, but they can be adapted for any RAID level the Linux kernel driver supports.

1. Create a new RAID array

Create (mdadm –create) is used to create a new array:
mdadm --create --verbose /dev/md0 --level=1 /dev/sda1 /dev/sdb2
Read the rest of this entry »

The Address Resolution Protocol (ARP) is the method for finding a host’s link layer (hardware) address when only its Internet Layer (IP) or some other Network Layer address is known. ARP is a Link Layer protocol (Layer 2) because it only operates on the local area network or point-to-point link that a host is connected to. When we migrate one IP from a machine to another one, we might have problems caused by ‘arp caching‘. Various devices will cache the arp information for a specified amount of time and even after we moved the IP this will not be seen by some devices that will still use the cached information. I am talking about directly connected switches or routers, that we might have control or maybe not. If we have control on all the external devices, normally we just connect to the router or switch and remove the arp entry, forcing the device to query again for the information. This post will try to help in the situation where we don’t have direct control on the external devices (we are collocated or use rented servers in a remote datacenter, etc.), to minimize the downtime associated with this type of IP migration.

It is quite frequent to use separate IPs for various services on the same machine, and move those IPs to another server if needed. These are sometimes called portable IPs that can be migrated to any server in a particular colo/lan. This is done normally to minimized downtime and keep maintenance of such operations minimal (and to not rely on dns changes). Still arp caching on various network devices can cause big problems. Let’s assume we moved the IP from one server to another one in the same LAN to move away some service from our main web server. Taking down the IP from the existing server and bringing it up on the new server will complete our direct work if we don’t have access on the switches/routers in front of us. Again if you have control on all devices just connect to them and delete the arp cache for this ip to allow it to be re-cached on the new machine.

Read the rest of this entry »

Awstats will consider as a page hit any entry from the log it processes. By default some file extensions (for regular image types and css/js) are excluded from what awstats will consider as a page:
NotPageList="css js class gif jpg jpeg png bmp ico"(this is the default). All other file types will be counted as pages. Now, if we want to completely ignore some files, or even all the content of one folder from the awstats processing we can use the SkipFiles parameter. We might want to do this to ignore some frames, hidden pages, ajax calls, etc.

Read the rest of this entry »

Sendmail will use by default a single mail queue. This is what most users will need, and if you don’t have any special requirement you will not care about this. Still for high traffic mail servers it might be useful to split the queue over several directories, as thousands of files in a single directory will become a performance penalty at some point and also processing the queue sequentially will become very slow. This post will show how we can implement multiple mail queues with modern sendmail versions.

Read the rest of this entry »

Anyone coming from the “sendmail world” to postfix will notice immediately the care taken by postfix developers to make the transition as easy as possible. The same commands you would be used to, like for ex. mailq, newaliases, are working in postfix also.

The newaliases command works on postfix as expected, rebuilding the aliases database:
newaliases

Read the rest of this entry »

This article will explain how you can install the latest memcached daemon (including the libevent library) on a linux system. The only prerequisite for memcached is libevent so we will have to install this first.

Note: the output of the commands in this article are taken from a Debian Etch system. They should work on any recent linux distribution, but depending from your version you might need to make some changes. The versions of memcached and libevent used in this article are the latest stable one existing at the time this was written. Check the download pages bellow, and if newer versions exists you will probably want to use them. Read the rest of this entry »