Building packages is a task that every system administrator will end up doing. Most of the time this is not a very interesting task but someone has to do it, right? Normally you will end up modifying and tweaking based on your own needs an existing package that was built by the maintainers of the Linux distribution that you are using. In time you might even become familiar with the packaging system you are using (rpm, deb, etc.) and you will be able to write a spec file and start from scratch and build a new package if you need to. Still, this process is complicated and requires a lot of work.

Luckily, Jordan Sissel has built a tool called FPM (Effing Package Management), exactly for this: to ease the pain of building new packages; packages that you will use for your own infrastructure and you want them customized based on your own needs; and you don’t care about upstream rules and standards and other limitations when building such packages. This can be very useful for people deploying their own applications as rpms (or debs) and can simplify a lot of the process of building those packages.

FPM can be easily installed on your build system using rubygems:
gem install fpm

Once installed you can use fpm to build packages (targets):

• deb
• rpm
• solaris

from any of the following sources:

• directory (of compiled source of some application)
• gem
• python eggs
• rpm
• node npm packages

Read the rest of this entry »

If you used vagrant (great tool, right?) you have probably downloaded a basebox from some remote location to get you started. This is a great quick start, and there are many good boxes out there that you can use; vagrantbox.es does a great job in listing various public vagrant boxes. But if you are like me, you probably will want to customize the boxes you are using; you might want to install them from scratch based on your own little/or/big customizations. Well if you are like that, then you will be happy to hear that Patrick Debois had exactly the same problem when he decided to write veewee. And veewee is exactly that missing part of vagrant that allows you to easily build your own vagrant boxes from scratch.

So let’s see how we can use veewee. I’m assuming you already have vagrant installed (and virtualbox), but if you don’t please install them first. To install veewee we just have to install the veewee gem:
gem install veewee
once you installed veewee you can see a new task added to vagrant: basebox.

Read the rest of this entry »

Recently I’ve worked on a project where we deployed a bunch KVM instances. Immediately we noticed horrible IO performance on all the guests instances. In this particular case the hosts and the guests were all Ubuntu 10.04 Lucid and were created with vmbuilder without any special settings using the ubuntu defaults. Here is a sample command similar to what we used to build the kvm images:

vmbuilder kvm ubuntu --suite=lucid --flavour=virtual --arch=amd64 --mirror=http://en.archive.ubuntu.com/ubuntu -o --libvirt=qemu:///system --ip=10.0.0.11 --gw=10.0.0.1 --part=vmbuilder.partition --templates=mytemplates --user=username --pass=password --firstboot=/var/vms/vm1/boot.sh --mem=1024 --hostname=myhost --bridge=br0

Now even if we haven’t tuned anything I would have expected it to perform at least the same level or even better compared with a Xen instance. Still, this was not the case, and the performance was really horrible and any kind of IO bound tasks would effectively lock the instance. Looking into this and trying to understand what was the problem I was able to isolate this issue happening only on instances that had ext4 as the filesystem (the default for lucid), but strangely enough this didn’t happen for an older instance that was build with ext3 (actually a debian lenny instance). All the images build with the above command will use qcow2 sparse format as the default format for the disk.

Read the rest of this entry »

A few days ago Opscode released a security fix for chef server 0.10.0 and 0.9.16 and this post will show how upgrade to chef-server 0.10.2. First start by backing up your data. Seriously. In the past I’ve had serious problems when performing similar upgrades (even a minor one like this that looks harmless), and even if now opscode are much better with this process it never hurts to be precautions. Since I use a rubygem install the next steps will focus on this type of installation; if you are using distribution or opscode packages this will not be very helpful as probably packages are not yet available for this upgrade; once they will replace the gem upgrade part with the deb/rpm upgrade and you should be set.

1. Stop all the chef related services

Here is a handy command that will stop all the possible chef server related services:
for svc in server server-webui solr expander
do
sudo /etc/init.d/chef-${svc} stop
done

Read the rest of this entry »

After installing a clean Debian Lenny Xen system using xen-tools, I received this strange error when trying to connect using ssh to the machine:
PTY allocation request failed on channel 0
stdin: is not a tty

It looks like for some reason, xen-tools didn’t install the udev package. So in order to fix this issue, I had to connect (using the virtual console) to the xen machine:
xen console 1
and install udev:
apt-get install udev
strange enough the /dev/pts mount entry was present in /etc/fstab so all I had to do was to remount it with:
mount -a
(if you don’t have this entry make sure to add it in /etc/fstab:
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=620 0 0
and if the folder /dev/pts doesn’t exist create it first and after that mount -a).

This should fix the ssh problem and you should now be able to ssh into the xen machine. Next to see if this is fixed in xen-tools in Squeeze, and if not to file a bug for it.

This post will show how to upgrade from Debian 5.0.x “Lenny” to the latest stable Debian release 6.0 “Squeeze”. One of the reasons I’ve liked Debian in the first place was the advantage of being able to do a live, in place updates from one major release to another, usually in a safe way. As always, if you do this, please take some time to backup your system if you care of your data, as this is a major upgrade and things can go wrong. Squeeze brings in a few big changes and I will outline some of them, but I would recommend to read the release notes and look for any incompatibilities (hardware or software) or changed things that could affect your particular setup.

1. Update apt sources.list

The first thing we will do (after the backup of course) is to edit the /etc/apt/sources.list file and replace “lenny” with “squeeze“. Originally, this might look like this (for a system using the main US mirrors; your file might use a different local one):

deb http://ftp.us.debian.org/debian/ lenny main contrib non-free
deb-src http://ftp.us.debian.org/debian/ lenny main contrib non-free

deb http://security.debian.org/ etch/updates lenny contrib non-free

after replacing lenny with squeeze the file will look like this:

deb http://ftp.us.debian.org/debian/ squeeze main contrib non-free
deb-src http://ftp.us.debian.org/debian/ squeeze main contrib non-free

deb http://security.debian.org/ squeeze/updates main contrib non-free

Read the rest of this entry »

Updated Debian GNU/Linux: 5.0.6 release

This week the Debian project released the 6th update to its stable release Lenny, 5.0.6. All recent security updates have been added, as well as some other fixes. The linux-2.6 package was also updated for increased hardware support.

Backports service is now official

I was very happy to hear that the debian backports project is now an official debian project. I always used (and liked) the backports.org repository to easily bring in updated software to the stable release. Now, after it become an official project and not just a fun project of three developers will hopefully be even better and have more software added into backports much faster. Don’t forget to change your apt sources config to point to backports.debian.org (old backports.org mirror will still work for a while).
deb http://backports.debian.org/debian-backports lenny-backports main contrib non-free

Debian growth over time

Also on some unrelated news Romain Francoise published some interesting stats on the growth of the Debian archive over time:

• woody (2002): 8273 packages
• sarge (2005): 15195 packages (+83.7%)
• etch (2007): 18043 packages (+18.7%)
• lenny (2009): 22277 packages (+23.5%)
• squeeze (2010?): 28870 packages (+29.6%)

Wow… now that is really impressive.

Squeeze has been frozen for some time now, and hopefully will be released by the end of the year, and today the Debian team has revealed the name of the next Debian release 7.0: Wheezy.

Just like all the previous releases, this is another character from Toy Story – wheezy – a rubber squeeze toy penguin with a red bow tie (that appears only in the 2nd movie). This will be the first character selected as a Debian version name which has not appeared in all the movies.

Source: http://lists.debian.org/debian-devel-announce/2010/09/msg00000.html

Everyone knows and loves screen for running longtime scripts in the background without worrying that the ssh connection will drop and will have to run it again. Still, I have found myself many times in the situation where I started a process and needed to put it in the background and run something else on the console. Uff.. If only I started it with screen. But wait, there is hope. This quick tip will show how to put a process in the background and then start it back in foreground.

This works in bash and uses the ‘suspend‘ key (CTRL+Z) and the bg – background and fg – foreground commands. Let’s say we were running an intensive rsync command, and are wanted to check if we still have the available space on the disk without opening a new ssh session (yes, I know):
rsync -ar server:/source/ /destination/
^Z
Stopped

Let it run in the background:
bg
[1] rsync -ar server:/source/ /destination/ &

Now we can run some other commands like du:
du -h

We can see the background process with ps or jobs:
jobs
[1] Running rsync -ar server:/source/ /destination/

And finally we can bring it back to foreground with fg:
fg

Note: this works only on the running ssh/bash session and it will be closed once you exit. Logout should warn about open/running jobs and that they will be lost if exit.

Debian has a nice way to handle multiple java installations on a the same machine. Let’s say that for some reason you want to have sun-java 1.5 and also 1.6 installed on the server, we can easily configure the default one with the update-java-alternatives command (part of the java-common package). Here is how it can be used:

To see what versions of java we have installed on the system (from debian packages):
update-java-alternatives -l
java-1.5.0-sun 53 /usr/lib/jvm/java-1.5.0-sun
java-6-sun 63 /usr/lib/jvm/java-6-sun

We can see that the default version is 1.6 in my case (as it was the last installed):
java -version
java version "1.6.0_20"
Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
Java HotSpot(TM) Client VM (build 16.3-b01, mixed mode, sharing)

We can change the default version with: update-java-alternatives –jre -s <ver> , like:
update-java-alternatives --jre -s java-1.5.0-sun
and now the default is 1.5:
java -version
java version "1.5.0_22"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_22-b03)
Java HotSpot(TM) Client VM (build 1.5.0_22-b03, mixed mode, sharing)

This is quite handy if you need to have multiple java versions installed, and need a quick way to change the default one (you can access any of them directly from their own path of course).