Building packages is a task that every system administrator will end up doing. Most of the time this is not a very interesting task but someone has to do it, right? Normally you will end up modifying and tweaking based on your own needs an existing package that was built by the maintainers of the Linux distribution that you are using. In time you might even become familiar with the packaging system you are using (rpm, deb, etc.) and you will be able to write a spec file and start from scratch and build a new package if you need to. Still, this process is complicated and requires a lot of work.

Luckily, Jordan Sissel has built a tool called FPM (Effing Package Management), exactly for this: to ease the pain of building new packages; packages that you will use for your own infrastructure and you want them customized based on your own needs; and you don’t care about upstream rules and standards and other limitations when building such packages. This can be very useful for people deploying their own applications as rpms (or debs) and can simplify a lot of the process of building those packages.

FPM can be easily installed on your build system using rubygems:
gem install fpm

Once installed you can use fpm to build packages (targets):

• deb
• rpm
• solaris

from any of the following sources:

• directory (of compiled source of some application)
• gem
• python eggs
• rpm
• node npm packages

Read the rest of this entry »

If you used vagrant (great tool, right?) you have probably downloaded a basebox from some remote location to get you started. This is a great quick start, and there are many good boxes out there that you can use; vagrantbox.es does a great job in listing various public vagrant boxes. But if you are like me, you probably will want to customize the boxes you are using; you might want to install them from scratch based on your own little/or/big customizations. Well if you are like that, then you will be happy to hear that Patrick Debois had exactly the same problem when he decided to write veewee. And veewee is exactly that missing part of vagrant that allows you to easily build your own vagrant boxes from scratch.

So let’s see how we can use veewee. I’m assuming you already have vagrant installed (and virtualbox), but if you don’t please install them first. To install veewee we just have to install the veewee gem:
gem install veewee
once you installed veewee you can see a new task added to vagrant: basebox.

Read the rest of this entry »

Ever since I sow the oscon presentation of reconnoiter I wanted to check it out and play with it. Yesterday, I finally had some time to do this and thought it would be a good idea to document it as a short howto. Most of the infos I used are from the readme (BUILDING), the wiki and the excellent writeup of Thomas Dudziak on how to install reconnoiter on ubuntu.

The daemons noitd and stratcond are written in C, and the database used is postgressql, while the web interface is written in php. We will need to install a few dependencies to be able to compile noitd/stratcond: Read the rest of this entry »

With the release of GoogleCL, the command line tool for the Google data APIs, Google reconfirmed if that was needed that it’s a geeky company (I mean you would not expect something like this form M$, right?) and they like command line tools. They released some basic command line tools for calendar, contacts, docs, picassa, blogger and youtube. Of course, coming from google the tools are written in their preferred language, python.

~$ google
> help
Welcome to the Google CL tool!
Commands are broken into several parts: service, task, options, and arguments.
For example, in the command
"> picasa post --title "My Cat Photos" photos/cats/*"
the service is "picasa", the task is "post", the single option is a name of "My Cat Photos", and the argument is the path to the photos.
The available services are 'picasa', 'blogger', 'youtube', 'docs', 'contacts', 'calendar'
Enter "> help <service>" for more information on a service.
Or, just "quit" to quit.

Read the rest of this entry »

The geoip iptables extension allows you to filter, nat or mangle packets based on the country’s source or destination. This does exactly what the geoip apache module does, or the regular geoip binary, but at the iptables level. I would not go into the details why you would want to use that, but there are many ‘positive’ ways it can be useful… For example myself I use it in a project where we want to serve customized content for different countries. Since this is a high traffic site running on many web servers behind a loadbalanced setup, we prefer to split this at the loadbalancer level and not at apache level, to simplify our setup. We serve customized content to the US based visitors, while for the other countries we serve another international site.

Now this has been working fine for a long time now, using the original geoip module and patch-o-matic-ng method of installation (similar to what is very well described here). Still, this is unmaintained, and starting with kernel 2.6.22 it is no longer working. There is a patch that will make it work with a newer kernel, but if you run iptables 1.4.x this will again fail and even if there are some manual walkarounds this is still not the best solution.

The solution is called Xtables-addons. Xtables-addons is the successor to patch-o-matic-ng. Likewise, it contains extensions that were not, or are not yet, accepted in the main kernel/iptables packages. Xtables-addons is different from patch-o-matic in that you do not have to patch or recompile the kernel, sometimes recompiling iptables is also not needed.
The latest version 1.12 supports: iptables >= 1.4.1 and kernel-source >= 2.6.17.

Read the rest of this entry »

I just finished the migration of my trac installation to redmine. The redmine migrate_from_trac documentation helped a lot, but there were some issues that didn’t work quite as described; this post will describe the correct steps I had to follow in order to import my old trac data (tickets, wiki pages, etc) in redmine.

1. Use Redmine 0.8.x stable

I lost some time with the current trunk version (r2571) until I realized that the importer is broken in trunk. Use the latest stable 0.8.x version, and if you want to use trunk, upgrade to it after you have the trac data inside redmine. While using trunk all was working fine (no error or anything strange), just the wiki pages were not imported in the redmine project. Anyway, I will assume you have a working clean installation of redmine 0.8.x before we will move on. Read the rest of this entry »

In my previous post, I introduced iotop a very cool tool that displays a table of current I/O usage by processes on the system; just as useful as top, but for I/O monitoring. Unfortunately, iotop requires Python >= 2.5 and a Linux kernel >= 2.6.20 to work, and even if the installation is very simple as presented in my last post, getting it to run on older distributions might not be so easy. This post will show how you can run iotop on debian etch, describing how we can solve the dependencies and make iotop run just fine on etch.
Read the rest of this entry »

iotop does for I/O usage what top does for CPU usage. It watches I/O usage information output by the Linux kernel (requires 2.6.20 or later) and displays a table of current I/O usage by processes on the system. This tool is written by Guillaume Chazarain and requires Python >= 2.5 and a Linux kernel >= 2.6.20 to run. This post introduces this very useful tool and shows how we can install it and use it.

iotop can be downloaded either as source package or a rpm package. Starting with lenny, debian includes iotop in the main repository and it can be installed just as simple as running:
aptitude install iotopThis is very cool indeed and kudos to the debian team to include iotop in lenny
Read the rest of this entry »

We all know and love vmstat, but wouldn’t it be nice to get such information on a per process basis, to be able to better understand what is causing i/o problems? This is exactly what iopp, written by Mark Wong and released as open source does:
“It’s a custom tool to go through the Linux process table to get i/o statistics per process. It is open source and can be downloaded from: http://git.postgresql.org/?p=~markwkm/iopp.git;a=summary“

Now this sounds interesting, and I am sure anyone that has dealt with i/o issues in the past will probably find this very useful. Let’s see how we can install it and what kind of reporting we get. We will install this from source and here are some quick steps to do this (you will need git and cmake for this):
git clone git://git.postgresql.org/git/~markwkm/iopp.git
cd iopp
cmake CMakeLists.txt
make

Read the rest of this entry »

Mdadm is the modern tool most Linux distributions use these days to manage software RAID arrays; in the past raidtools was the tool we have used for this. This cheat sheet will show the most common usages of mdadm to manage software raid arrays; it assumes you have a good understanding of software RAID and Linux in general, and it will just explain the commands line usage of mdadm. The examples bellow use RAID1, but they can be adapted for any RAID level the Linux kernel driver supports.

1. Create a new RAID array

Create (mdadm –create) is used to create a new array:
mdadm --create --verbose /dev/md0 --level=1 /dev/sda1 /dev/sdb2
Read the rest of this entry »