Many times you might want to fine tune the default permissions of the files created on a linux system. This is very simple and usually if you are using bash all you have to do is to define somewhere in the bash startup files (/etc/profile is a good place for this) a new value for umask like this:
umask 002
(this will allow by default group write permissions on the newly created files)

Normally on modern linux distributions this is by default set to 022 and you can easily find out what it is on your system by running the umask command:
umask

Contrary to what you might think, this is not enough to have this working for all applications and daemons on the system. This works fine for any files created from a shell session, but the files created by other processes, like the web server for example, will still use the default, unless otherwise configured. In order to have apache use a different umask we can define this inside /etc/apache2/envvars (debian, and ubuntu systems) or /etc/sysconfig/httpd (rhel,centos systems) like this:
umask 002
and restart apache to enable it.

Other daemons will have different locations where you can define this to overwrite the default setting for umask (check their documentation if you are unsure).

I needed to get a list of all the subdirectories that were owner by some other user than root under /var and their permissions/owner with full paths. My first thought was to use ls and something like this:
ls -dlR */
drwxr-xr-x  2 root root  4096 2009-06-05 06:25 backups/
drwxr-xr-x  8 root root  4096 2009-05-11 06:02 cache/
drwxr-xr-x  2 root root  4096 2009-05-06 04:49 ec2/
drwxr-xr-x 25 root root  4096 2009-05-25 14:55 lib/
...
will show the subdirectories just as I needed but only at one level. Using */*/ would show the next level, etc. This obviously is not a solution and unfortunately I had found no other way to do this with ls. Using:
ls -alR | grep ^d
drwxr-xr-x 15 root root  4096 2009-05-11 06:02 .
drwxr-xr-x 22 root root  4096 2009-06-03 15:02 ..
drwxr-xr-x  2 root root  4096 2009-06-05 06:25 backups
drwxr-xr-x  8 root root  4096 2009-05-11 06:02 cache
drwxr-xr-x  2 root root  4096 2009-05-06 04:49 ec2
drwxr-xr-x 25 root root  4096 2009-05-25 14:55 lib
....
works somehow, but since I don’t have the full paths this is useless.

Read the rest of this entry »

Here is a quick tip that will show how you can get a small sample dataset from a mysql database using mysqldump. We frequently need to get a small snapshot from a very big production database to import it into a development or staging database that will not need all the original data; let’s say we need 1,000,000 records from all the tables in the database; we will just use the option –where=”true LIMIT X”, with X the number of records we want mysqldump to stop after.

Simply we will run something like (add whatever other options you need to mysqldump):

mysqldump --opt --where="true LIMIT 1000000" mydb > mydb1M.sql
Read the rest of this entry »

Mdadm is the modern tool most Linux distributions use these days to manage software RAID arrays; in the past raidtools was the tool we have used for this. This cheat sheet will show the most common usages of mdadm to manage software raid arrays; it assumes you have a good understanding of software RAID and Linux in general, and it will just explain the commands line usage of mdadm. The examples bellow use RAID1, but they can be adapted for any RAID level the Linux kernel driver supports.

1. Create a new RAID array

Create (mdadm –create) is used to create a new array:
mdadm --create --verbose /dev/md0 --level=1 /dev/sda1 /dev/sdb2
Read the rest of this entry »

The Address Resolution Protocol (ARP) is the method for finding a host’s link layer (hardware) address when only its Internet Layer (IP) or some other Network Layer address is known. ARP is a Link Layer protocol (Layer 2) because it only operates on the local area network or point-to-point link that a host is connected to. When we migrate one IP from a machine to another one, we might have problems caused by ‘arp caching‘. Various devices will cache the arp information for a specified amount of time and even after we moved the IP this will not be seen by some devices that will still use the cached information. I am talking about directly connected switches or routers, that we might have control or maybe not. If we have control on all the external devices, normally we just connect to the router or switch and remove the arp entry, forcing the device to query again for the information. This post will try to help in the situation where we don’t have direct control on the external devices (we are collocated or use rented servers in a remote datacenter, etc.), to minimize the downtime associated with this type of IP migration.

It is quite frequent to use separate IPs for various services on the same machine, and move those IPs to another server if needed. These are sometimes called portable IPs that can be migrated to any server in a particular colo/lan. This is done normally to minimized downtime and keep maintenance of such operations minimal (and to not rely on dns changes). Still arp caching on various network devices can cause big problems. Let’s assume we moved the IP from one server to another one in the same LAN to move away some service from our main web server. Taking down the IP from the existing server and bringing it up on the new server will complete our direct work if we don’t have access on the switches/routers in front of us. Again if you have control on all devices just connect to them and delete the arp cache for this ip to allow it to be re-cached on the new machine.

Read the rest of this entry »

Here is a quick bash tip that might be useful if you need to use inside a bash script a check to see if a wildcard expression of files/folders exists or not. For example:

if [ -e /tmp/*.cache ]
then
echo "Cache files exist: do something with them"
else
echo "No cache files..."
fi
This is using -e (existing file check) that is working fine on individual files. Also the above code might seem to work fine if the result of the expression if one file; but if you have more files returned by the expression this will fail with the following error:
line x: [: too many arguments

Read the rest of this entry »

This post will show how to deal with the issue I had on a newly installed debian lenny xen virtual machine. I used xen-tools to create the vm using the deboostrap method, and all was fine. I installed the bash-completion package, as probably most of you, I can’t live without bash completion, but quickly found out that it was broken. Any attempt to perform a filelist completion was failing with this error:
vm11:~# tail -f /va<TAB>-bash: /dev/fd/62: No such file or directory
-bash: /dev/fd/60: No such file or directory
and this basically makes the bash completion useless. On a quick look I could easily see that the /dev/fd link was not there and that was the main cause of the problem. Still on one older lenny vm I had for a couple of months this was not happening (from what I can tell because it had an older version of the /etc/bash_completion file). There are several ways to fix this starting with the obvious one to downgrade /etc/bash_completion but I didn’t like that, so I looked for some other ways. Read the rest of this entry »

There are various methods to run multiple instances of mysql (on different ports) on the same machine. We can either compile the mysql binary with different defaults and paths, use mysqld_multi or the MySQL Sandbox project. Still the simplest solution I’ve  used in the past for such situations is to use the same binary and use a separate configuration file (with separate port, pid, socket and data directory). This post will explain this method. Read the rest of this entry »

For grep expressions the OR operator is “\|“. Here is an example using tail on a file looking for 2 strings occurrences:
tail -f /var/log/apache2/error.log | grep "PHP Notice\|client denied"
or another example to list the content of the php.ini file without comments and empty lines:
grep -v "^#\|^$\|^;" /etc/php5/apache2/php.ini

The % character is a special char for crontab entries, and in order to use it we have to escape it. For example a crontab entry like this:
1 0 * * * /bin/sleep `/usr/bin/expr $RANDOM % 600` ; /usr/local/bin/mycommand
that attempts to run mycommand hourly with a random delay for the start, will fail because cron will see the % character and ignore the rest of the command; it will just run: /bin/sleep `/usr/bin/expr $RANDOM that will return a random number between 0-32767 and exit.

Read the rest of this entry »