The Debian project just announced the third update for its stable distribution “lenny” 5.0.3. Those installing regular updates from security.debian.org will notice just a few new updates (base-files for the version change to 5.0.3, heartbeat, perl, openssl, linux-image, svn, etc). Also the installer has been updated to incorporate the new kernels released with this point release, adding support for new network hardware, and to fix a segfault early in the boot process of installations for the S/390 architecture.

“The Debian project is pleased to announce the third update of its stable distribution Debian GNU/Linux 5.0 (codename lenny). This update mainly adds corrections for security problems to the stable release, along with a few adjustment to serious problems.

Please note that this update does not constitute a new version of Debian GNU/Linux 5.0 but only updates some of the packages included. There is no need to throw away 5.0 CDs or DVDs but only to update via an up-to-date Debian mirror after an installation, to cause any out of date packages to be updated.

Those who frequently install updates from security.debian.org won’t have to update many packages and most updates from security.debian.org are included in this update.

New CD and DVD images containing updated packages and the regular installation media accompanied with the package archive respectively will be available soon at the regular locations.”

Release Announcement: http://www.debian.org/News/2009/20090905

Amazon just announced the limited beta of Amazon Virtual Private Cloud (Amazon VPC), a secure and seamless bridge between existing IT infrastructures and the AWS cloud. Amazon VPC enables us to connect our existing infrastructure to a set of isolated AWS compute resources via a Virtual Private Network (VPN) connection.

“Amazon VPC enables you to use your own isolated resources within the AWS cloud, and then connect those resources directly to your own datacenter using industry-standard encrypted IPsec VPN connections. With Amazon VPC, you can:

• Create a Virtual Private Cloud on AWS’s scalable infrastructure, and specify its private IP address range from any block you choose.
• Divide your VPC’s private IP address range into one or more subnets in a manner convenient for managing applications and services you run in your VPC.
• Bridge together your VPC and your IT infrastructure via an encrypted VPN connection.
• Add AWS resources, such as Amazon EC2 instances, to your VPC.
• Route traffic between your VPC and the Internet over the VPN connection so that it can be examined by your existing security and networking assets before heading to the public Internet.
• Extend your existing security and management policies within your IT infrastructure to your VPC as if they were running within your infrastructure.”

Besides the regular ec2 prices, we will have to pay for the VPN connection ($0.05 per VPN Connection-hour) and for the data transfer using the VPN tunnel ($0.10 per GB IN – and starting with $0.17 per GB OUT).

This is a great new service from Amazon that takes the cloud computing offerings to a new level. And this just when major competitors thought they were gaining on Amazon with their offerings, Amazon continues to innovate and launch great new services like this one .

I’ve just finished reading “Learning Nagios 3.0″ by Wojciech Kocjan and published by Packt Publishing, and this is a great book for anyone interested in nagios. This is a beginner level book that introduces nagios to new users interested in monitoring their infrastructure, but it will also present advanced features that even more experienced sysadmins can benefit from. All these in a pretty compact book, at 301 pages.

The topics are as follows:

• Introduction
• Installation and Configuration
• Using the Nagios Web Interface
• Overview of Nagios Plugins
• Advanced Configuration
• Notifications and Events
• Passive Checks and NSCA
• Monitoring Remote Hosts
• SNMP
• Advanced Monitoring
• Extending Nagios

Read the rest of this entry »

One of the great features of Amazon EC2 is the possibility to dynamically query and use instance specific metadata, or even custom data. This can be useful for various reasons, and the greatest advantage I’ve personally seen into this, is the possibility to allow the instance to have some information on how to configure itself when first booting (using chef or puppet, or some other configuration management tool).

The Amazon documentation explains how to get this information, basically just by using simple http get requests on the ip: 169.254.169.254, like for ex (for the metadata index):
curl http://169.254.169.254/latest/meta-data/
or for the custom data:
curl http://169.254.169.254/latest/user-data

Eucalyptus supports this great feature (starting with v1.4), but we obviously need to target a different ip to retrieve this information (as the amazon ip has nothing to do with our internal cloud ). We need to use the cloud controller IP for the request and the port it is bound (by default 8773 if you have not changed it). This will look like this (you need to run it from inside the actual instance): Read the rest of this entry »

s3sync is a great tool to synchronize local data with Amazon S3 for backups, or whatever other reasons you might want to put your data on S3. It is very simple to install (gem install s3sync) and use (s3sync -v -s -r –progress <source_dir> s3_bucket:<dir>); it runs very well and it can be easily scripted to do regular backups or even synchronize live data with S3. The only problem I found while using s3sync was that it can be very slow when uploading a lot of data (millions of files) to S3; this because the process is slow but also because it runs a single file at a time, and it doesn’t do several uploads in parallel. I would have loved for s3sync to do this out of the box, but unfortunately it doesn’t, but for my particular need I was able to do this by running more s3sync commands a the same time. It will not apply to your data (unless it is structured the same way as here; very unlikely), but it might give an idea on how you could do this your own data if it is structured in a feasible way.

Read the rest of this entry »

Many startups these days are using Amazon S3 to serve directly their static assets. S3 is being used as a simple CDN instead of more professional (and expensive) solutions (including Amazon’s own CloudFront) because it is very simple and cheap to use. Still if you have a high traffic site, this will no longer be so cheap since you will be paying for all those requests and the bandwidth. In such cases if you still want to use S3 for the storage advantage (like storing millions of files and see it as an unlimited storage space) but not have your bill go up like crazy, you can use a reverse proxy or web accelerator to cache your assets locally and reduce the number of direct hits on S3. We could use Squid or Varnish for this, and in this article I will show how we can configure Varnish for this. We are using varnish with S3 on various projects and it works very well, simplifying the setup and saving a lot of money in the Amazon S3 bill.

Varnish is a state-of-the-art, high-performance HTTP accelerator. It uses the advanced features in Linux 2.6, FreeBSD 6/7 and Solaris 10 to achieve its high performance. I will not go over the installation of varnish here, but I would highly recommend to use the latest version available at this time 2.0.4 as older versions have various issues.

We could try to use something simple like this in a varnish vcl:

backend s3 {
   set backend.host = "my_bucket.s3.amazonaws.com";
   set backend.port = "80";
}

sub vcl_recv {
   if (req.url ~ "\.(css|gif|ico|jpg|jpeg|js|png|swf|txt)$") {
     set req.backend = s3;
     lookup;
   }
}

Read the rest of this entry »

Many times you might want to fine tune the default permissions of the files created on a linux system. This is very simple and usually if you are using bash all you have to do is to define somewhere in the bash startup files (/etc/profile is a good place for this) a new value for umask like this:
umask 002
(this will allow by default group write permissions on the newly created files)

Normally on modern linux distributions this is by default set to 022 and you can easily find out what it is on your system by running the umask command:
umask

Contrary to what you might think, this is not enough to have this working for all applications and daemons on the system. This works fine for any files created from a shell session, but the files created by other processes, like the web server for example, will still use the default, unless otherwise configured. In order to have apache use a different umask we can define this inside /etc/apache2/envvars (debian, and ubuntu systems) or /etc/sysconfig/httpd (rhel,centos systems) like this:
umask 002
and restart apache to enable it.

Other daemons will have different locations where you can define this to overwrite the default setting for umask (check their documentation if you are unsure).

Earlier this week, at DebConf 9, the Debian team proposed a new approach for the Debian’s release cycle, which was later on announced publicly on the Debian site:

“The Debian project has decided to adopt a new policy of time-based development freezes for future releases, on a two-year cycle. Freezes will from now on happen in the December of every odd year, which means that releases will from now on happen sometime in the first half of every even year. To that effect the next freeze will happen in December 2009, with a release expected in spring 2010. The project chose December as a suitable freeze date since spring releases proved successful for the releases of Debian GNU/Linux 4.0 (codenamed “Etch”) and Debian GNU/Linux 5.0 (“Lenny”).”

This doesn’t mean that we will have a time-based release as for example Ubuntu does on a specific date, but it means that we will have a time-based freeze for each new release (in December of every odd year); the release will still become stable “when it is ready”, but after this, we can expect the new releases in general sometimes in the spring of the every even year.

“Time-based freezes will allow the Debian Project to blend the predictability of time based releases with its well established policy of feature based releases. The new freeze policy will provide better predictability of releases for users of the Debian distribution, and also allow Debian developers to do better long-term planning. A two-year release cycle will give more time for disruptive changes, reducing inconveniences caused for users. Having predictable freezes should also reduce overall freeze time.”

This new approach will leave a very short time for the next release Debian 6.0 (“Squeeze”), that will be freezed later on this year (lenny was released earlier this year in February). Here are the major release goals for squeeze: multi-arch support, which will improve the installation of 32 bit packages on 64 bit machines, and an optimised boot process for better boot performance and reliability.

The moment I heard about the Open Source Bridge Configuration Management panel session on FLOSS Weekly a while ago, I was hoping that I will be able to see the recording of this session (as for obvious reasons I was not able to attend and see this live in Portland, Oregon). They managed to bring together (for the first time to my knowledge) the creators (or maintainers) of *all* the major configuration management tools to date was very impressive; and obviously someone as myself that has been working with many of these tools (I haven’t tried/used automateit yet) would definitely see this as a great session.

Here are the members of the configuration management panel (from left to right):

• Igal Koshevoy of AutomateIt
• Brendan Strejcek of Cfengine
• Luke Kanies from Reductive Labs for Puppet
• Narayan Desai of bcfg2
• Adam Jacob from Opscode for Chef

Read the rest of this entry »

This post is sponsored by FindMyHosting – a free and very comprehensive web hosting directory featuring the most popular web hosting companies and thousands of customer reviews.

I’ve been asked to review this site and give my impressions about it. The truth is that I don’t have much experience with shared hosting as most of my experience is with dedicated servers from various hosting companies, and anytime I had a friend asking about where do I recommend him to host his small site I didn’t knew where to direct him. This is why I thought that such a webhosting directory as FindMyHosting would be a great start for anyone looking for a shared hosting account to host his new site. We can search from a long list of hosting company and get them ranked by users reports (nice). Read the rest of this entry »