Building packages is a task that every system administrator will end up doing. Most of the time this is not a very interesting task but someone has to do it, right? Normally you will end up modifying and tweaking based on your own needs an existing package that was built by the maintainers of the Linux distribution that you are using. In time you might even become familiar with the packaging system you are using (rpm, deb, etc.) and you will be able to write a spec file and start from scratch and build a new package if you need to. Still, this process is complicated and requires a lot of work.

Luckily, Jordan Sissel has built a tool called FPM (Effing Package Management), exactly for this: to ease the pain of building new packages; packages that you will use for your own infrastructure and you want them customized based on your own needs; and you don’t care about upstream rules and standards and other limitations when building such packages. This can be very useful for people deploying their own applications as rpms (or debs) and can simplify a lot of the process of building those packages.

FPM can be easily installed on your build system using rubygems:

1

gem install fpm

Once installed you can use fpm to build packages (targets):

• deb
• rpm
• solaris

from any of the following sources:

• directory (of compiled source of some application)
• gem
• python eggs
• rpm
• node npm packages

Yesterday Opscode, the company behind Chef, announced the first ever chef cookbook contest. In order to participate in the contest you will need to write a new cookbook and submit it by the end of September; this is going to be a little tricky as there are many cookbooks already available on the community site. So this is a great idea and it will take care of the few applications that don’t already have chef cookbooks. The cookbooks which shows off the awesome Chef features will have better chances to win. The prizes are also interesting: iPad, gift cards, etc. Here are the full details and rules of the contest: http://www.opscode.com/blog/2011/08/22/cookbook-contest/

So if you have an idea for a chef cookbook, now it’s the time to start working on it. I’m offering my help for free for all my blog readers: I will help you write a cookbook by implementing your ideas; help reviewing it or suggest improvements, or whatever else you might need help with. Use the contact form to email me (or DM me on twitter) and let me know how I can help.

If you don’t have time to write a new cookbook but you have a great idea for a cookbook that is missing from the opscode community site, please post it bellow in the comments section and I’m sure some of my blog readers will help create it.

Again this is a brilliant idea from Opscode and it creates a win-win situation for everyone. I’m just curious, is this the first idea from their new community manager? If this is the case, great job Jesse ;).

If you used vagrant (great tool, right?) you have probably downloaded a basebox from some remote location to get you started. This is a great quick start, and there are many good boxes out there that you can use; vagrantbox.es does a great job in listing various public vagrant boxes. But if you are like me, you probably will want to customize the boxes you are using; you might want to install them from scratch based on your own little/or/big customizations. Well if you are like that, then you will be happy to hear that Patrick Debois had exactly the same problem when he decided to write veewee. And veewee is exactly that missing part of vagrant that allows you to easily build your own vagrant boxes from scratch.

So let’s see how we can use veewee. I’m assuming you already have vagrant installed (and virtualbox), but if you don’t please install them first. To install veewee we just have to install the veewee gem:

1

gem install veewee

once you installed veewee you can see a new task added to vagrant: basebox.

Yesterday evening I presented at the SF Bay Area Large-Scale Production Engineering meetup group at Yahoo HQ a talk about “Monitoring with Icinga”. This was an introductory talk intended to bring awareness about icinga (there were only 3-4 people from the audience of about 75 that heard of it before), and I think it reached its goal very well; afterwards there were many people interested to try it out and had various questions about it at the end. I was also very happy to have Matthew Brooks one of the icinga core developers in the audience and backing me up to some of the more difficult questions people had. Thanks again Matthew for coming! Here are the slides from my presentation:

Monitoring with Icinga @ SF Bay Area LSPE meetup

View more presentations from mdxp

@LSPEMeetup made available the video on justin.tv; unfortunately the quality of the video/sound is not the best; you can find it here.

Recently I’ve worked on a project where we deployed a bunch KVM instances. Immediately we noticed horrible IO performance on all the guests instances. In this particular case the hosts and the guests were all Ubuntu 10.04 Lucid and were created with vmbuilder without any special settings using the ubuntu defaults. Here is a sample command similar to what we used to build the kvm images:

1

vmbuilder kvm ubuntu --suite=lucid --flavour=virtual --arch=amd64 --mirror=http://en.archive.ubuntu.com/ubuntu -o --libvirt=qemu:///system --ip=10.0.0.11 --gw=10.0.0.1 --part=vmbuilder.partition --templates=mytemplates --user=username --pass=password --firstboot=/var/vms/vm1/boot.sh --mem=1024 --hostname=myhost --bridge=br0

Now even if we haven’t tuned anything I would have expected it to perform at least the same level or even better compared with a Xen instance. Still, this was not the case, and the performance was really horrible and any kind of IO bound tasks would effectively lock the instance. Looking into this and trying to understand what was the problem I was able to isolate this issue happening only on instances that had ext4 as the filesystem (the default for lucid), but strangely enough this didn’t happen for an older instance that was build with ext3 (actually a debian lenny instance). All the images build with the above command will use qcow2 sparse format as the default format for the disk.

A few days ago Opscode released a security fix for chef server 0.10.0 and 0.9.16 and this post will show how upgrade to chef-server 0.10.2. First start by backing up your data. Seriously. In the past I’ve had serious problems when performing similar upgrades (even a minor one like this that looks harmless), and even if now opscode are much better with this process it never hurts to be precautions. Since I use a rubygem install the next steps will focus on this type of installation; if you are using distribution or opscode packages this will not be very helpful as probably packages are not yet available for this upgrade; once they will replace the gem upgrade part with the deb/rpm upgrade and you should be set.

1. Stop all the chef related services

Here is a handy command that will stop all the possible chef server related services:

1
2
3
4

for svc in server server-webui solr expander
do
sudo /etc/init.d/chef-${svc} stop
done

Just like last year, after Velocity, I attended DevOpsDays in Mountain View, now at its second edition. Many people that were in town for Velocity spent at least one extra day for DevOpsDays, but also many people from the area that for some reason could not attend Velocity came to LinkedIn HQ for this event; about 200-250 people I would say. Even if the same type of people were at both events (awesome dev/ops), because of the completely different way they are organized they didn’t overlap at all. As you will see bellow in my impressions DevOpsDays wins largely in most of the areas:

• first of all it is a free event (compared with a regular O’Reilly conference where prices usually start at $1k).

• it is much more interactive: while Velocity is a classic conference where you normally have a presenter showing off something (hopefully not selling or hiring), and maybe some questions at the end, DevOpsDays is more like an open discussion, with people either on a panel or open spaces.

• the food was way better at DevOpsDays, no question about it. And the ice cream on Saturday added an extra special touch ;).

The first day, Friday, started with the ”Devops State of the Union” by John Willis. This was a very good introduction on what DevOps means and a look back on what happened during the past couple of years, especially considering the fact that many people where there for the first time. For example I met someone from Microsoft that was sent here to find out “what is this devops thing” and how they can use it, and this just shows what a huge progress the devops movement has made in such a short amount of time, and how many people are now interested in the movement. (in this particular case I’m not sure he returned at Microsoft with something useful, but just the fact that they are interested in this demonstrates my point).

Last week I attended Velocity 2011 in Santa Clara, CA; this was the 4th edition of Velocity and my 2nd one. This means that even if I was still very much impressed with everything what happened during these 3 days, it was not quite so mind blowing like last year, and you will be able to see this from my impressions bellow.

First impression was that this was much bigger than last year, and the numbers just released by O’Reilly demonstrate it was almost double with about 2,000 participants and another sell out (compared with 1,200 in 2010). Considering the high price of the conference (~1k) this is very impressive and interesting to see how they will host next year even more people. From a question a speaker had many people where at their first Velocity, but even so I met many friends and people I met last year. From the content point of view there was a lot of focus on mobile performance that I haven’t seen last year. Myself I mostly followed the operations track, and only a few talks in the performance track.

This was by far the best Ignite event I’ve ever attended. This was huge… packed, with more than 400 people in the room, and even if the room was huge it felt like there was no empty seat. Second, the quality and energy of the speakers was amazing. Most of the videos are already uploaded to youtube, and I would recommend to check them out if you have the time. By far my favorite one was Dr. Nic Williams - Ignite Karaoke, where he improvised on a series of “interesting” slides (who selected them?) that he never sow before. Amazing… Check it out:

My other favorites ones were: Ben Scofield - How to be Awesome and Gregg Pollack - How to Create Awesome Screencasts.

As my friends know I’m a big fan of Ingite) presentations, and the ones from MySQLConf 2011 last week were just outstanding. If you didn’t had the chance to see them live, here are the links to the best of them.

This was by far my favorite: “Scale Fail” by Josh Berkus; extremely funny on how to build sites that don’t scale. Highly recommended:

The other two were great ignite talks also, and definitely worth the time checking out: “Causes of Downtime in MySQL” by Baron Schwartz and “The Art of Data Visualization” by David Holoboff.