Ever since I sow the oscon presentation of reconnoiter I wanted to check it out and play with it. Yesterday, I finally had some time to do this and thought it would be a good idea to document it as a short howto. Most of the infos I used are from the readme (BUILDING), the wiki and the excellent writeup of Thomas Dudziak on how to install reconnoiter on ubuntu.

The daemons noitd and stratcond are written in C, and the database used is postgressql, while the web interface is written in php. We will need to install a few dependencies to be able to compile noitd/stratcond: Read the rest of this entry »

With the release of GoogleCL, the command line tool for the Google data APIs, Google reconfirmed if that was needed that it’s a geeky company (I mean you would not expect something like this form M$, right?) and they like command line tools. They released some basic command line tools for calendar, contacts, docs, picassa, blogger and youtube. Of course, coming from google the tools are written in their preferred language, python.

~$ google
> help
Welcome to the Google CL tool!
Commands are broken into several parts: service, task, options, and arguments.
For example, in the command
"> picasa post --title "My Cat Photos" photos/cats/*"
the service is "picasa", the task is "post", the single option is a name of "My Cat Photos", and the argument is the path to the photos.
The available services are 'picasa', 'blogger', 'youtube', 'docs', 'contacts', 'calendar'
Enter "> help <service>" for more information on a service.
Or, just "quit" to quit.

Read the rest of this entry »

The geoip iptables extension allows you to filter, nat or mangle packets based on the country’s source or destination. This does exactly what the geoip apache module does, or the regular geoip binary, but at the iptables level. I would not go into the details why you would want to use that, but there are many ‘positive’ ways it can be useful… For example myself I use it in a project where we want to serve customized content for different countries. Since this is a high traffic site running on many web servers behind a loadbalanced setup, we prefer to split this at the loadbalancer level and not at apache level, to simplify our setup. We serve customized content to the US based visitors, while for the other countries we serve another international site.

Now this has been working fine for a long time now, using the original geoip module and patch-o-matic-ng method of installation (similar to what is very well described here). Still, this is unmaintained, and starting with kernel 2.6.22 it is no longer working. There is a patch that will make it work with a newer kernel, but if you run iptables 1.4.x this will again fail and even if there are some manual walkarounds this is still not the best solution.

The solution is called Xtables-addons. Xtables-addons is the successor to patch-o-matic-ng. Likewise, it contains extensions that were not, or are not yet, accepted in the main kernel/iptables packages. Xtables-addons is different from patch-o-matic in that you do not have to patch or recompile the kernel, sometimes recompiling iptables is also not needed.
The latest version 1.12 supports: iptables >= 1.4.1 and kernel-source >= 2.6.17.

Read the rest of this entry »

I just finished the migration of my trac installation to redmine. The redmine migrate_from_trac documentation helped a lot, but there were some issues that didn’t work quite as described; this post will describe the correct steps I had to follow in order to import my old trac data (tickets, wiki pages, etc) in redmine.

1. Use Redmine 0.8.x stable

I lost some time with the current trunk version (r2571) until I realized that the importer is broken in trunk. Use the latest stable 0.8.x version, and if you want to use trunk, upgrade to it after you have the trac data inside redmine. While using trunk all was working fine (no error or anything strange), just the wiki pages were not imported in the redmine project. Anyway, I will assume you have a working clean installation of redmine 0.8.x before we will move on. Read the rest of this entry »

In my previous post, I introduced iotop a very cool tool that displays a table of current I/O usage by processes on the system; just as useful as top, but for I/O monitoring. Unfortunately, iotop requires Python >= 2.5 and a Linux kernel >= 2.6.20 to work, and even if the installation is very simple as presented in my last post, getting it to run on older distributions might not be so easy. This post will show how you can run iotop on debian etch, describing how we can solve the dependencies and make iotop run just fine on etch.
Read the rest of this entry »

iotop does for I/O usage what top does for CPU usage. It watches I/O usage information output by the Linux kernel (requires 2.6.20 or later) and displays a table of current I/O usage by processes on the system. This tool is written by Guillaume Chazarain and requires Python >= 2.5 and a Linux kernel >= 2.6.20 to run. This post introduces this very useful tool and shows how we can install it and use it.

iotop can be downloaded either as source package or a rpm package. Starting with lenny, debian includes iotop in the main repository and it can be installed just as simple as running:
aptitude install iotopThis is very cool indeed and kudos to the debian team to include iotop in lenny
Read the rest of this entry »

We all know and love vmstat, but wouldn’t it be nice to get such information on a per process basis, to be able to better understand what is causing i/o problems? This is exactly what iopp, written by Mark Wong and released as open source does:
“It’s a custom tool to go through the Linux process table to get i/o statistics per process. It is open source and can be downloaded from: http://git.postgresql.org/?p=~markwkm/iopp.git;a=summary“

Now this sounds interesting, and I am sure anyone that has dealt with i/o issues in the past will probably find this very useful. Let’s see how we can install it and what kind of reporting we get. We will install this from source and here are some quick steps to do this (you will need git and cmake for this):
git clone git://git.postgresql.org/git/~markwkm/iopp.git
cd iopp
cmake CMakeLists.txt
make

Read the rest of this entry »

Mdadm is the modern tool most Linux distributions use these days to manage software RAID arrays; in the past raidtools was the tool we have used for this. This cheat sheet will show the most common usages of mdadm to manage software raid arrays; it assumes you have a good understanding of software RAID and Linux in general, and it will just explain the commands line usage of mdadm. The examples bellow use RAID1, but they can be adapted for any RAID level the Linux kernel driver supports.

1. Create a new RAID array

Create (mdadm –create) is used to create a new array:
mdadm --create --verbose /dev/md0 --level=1 /dev/sda1 /dev/sdb2
Read the rest of this entry »

Trac uses an internal database to store its data, meaning some extra care needs to be taken when backing up a trac project. This means that we can’t just copy/tar/gz/etc the folder and we need to make sure the database is properly closed and no one is writing to it at that particular time. Of course this is useful for big installation, while for a small trac usage install, you can probably just tar gz the trac folder and be just fine. But even in that case, I would choose the safe method and do it right because it is so easy: the trac-admin command provides the hotcopy switch just for this.

Basically all you have to do is:
trac-admin <trac_project_path> hotcopy <backup_path>
and of course if you are familiar with svn you will notice that it is similar to the way you use svnadmin hotcopy
Read the rest of this entry »

Awstats will consider as a page hit any entry from the log it processes. By default some file extensions (for regular image types and css/js) are excluded from what awstats will consider as a page:
NotPageList="css js class gif jpg jpeg png bmp ico"(this is the default). All other file types will be counted as pages. Now, if we want to completely ignore some files, or even all the content of one folder from the awstats processing we can use the SkipFiles parameter. We might want to do this to ignore some frames, hidden pages, ajax calls, etc.

Read the rest of this entry »