Amazon just announced the limited beta of Amazon Virtual Private Cloud (Amazon VPC), a secure and seamless bridge between existing IT infrastructures and the AWS cloud. Amazon VPC enables us to connect our existing infrastructure to a set of isolated AWS compute resources via a Virtual Private Network (VPN) connection.

“Amazon VPC enables you to use your own isolated resources within the AWS cloud, and then connect those resources directly to your own datacenter using industry-standard encrypted IPsec VPN connections. With Amazon VPC, you can:

• Create a Virtual Private Cloud on AWS’s scalable infrastructure, and specify its private IP address range from any block you choose.
• Divide your VPC’s private IP address range into one or more subnets in a manner convenient for managing applications and services you run in your VPC.
• Bridge together your VPC and your IT infrastructure via an encrypted VPN connection.
• Add AWS resources, such as Amazon EC2 instances, to your VPC.
• Route traffic between your VPC and the Internet over the VPN connection so that it can be examined by your existing security and networking assets before heading to the public Internet.
• Extend your existing security and management policies within your IT infrastructure to your VPC as if they were running within your infrastructure.”

Besides the regular ec2 prices, we will have to pay for the VPN connection ($0.05 per VPN Connection-hour) and for the data transfer using the VPN tunnel ($0.10 per GB IN – and starting with $0.17 per GB OUT).

This is a great new service from Amazon that takes the cloud computing offerings to a new level. And this just when major competitors thought they were gaining on Amazon with their offerings, Amazon continues to innovate and launch great new services like this one .

One of the great features of Amazon EC2 is the possibility to dynamically query and use instance specific metadata, or even custom data. This can be useful for various reasons, and the greatest advantage I’ve personally seen into this, is the possibility to allow the instance to have some information on how to configure itself when first booting (using chef or puppet, or some other configuration management tool).

The Amazon documentation explains how to get this information, basically just by using simple http get requests on the ip: 169.254.169.254, like for ex (for the metadata index):
curl http://169.254.169.254/latest/meta-data/
or for the custom data:
curl http://169.254.169.254/latest/user-data

Eucalyptus supports this great feature (starting with v1.4), but we obviously need to target a different ip to retrieve this information (as the amazon ip has nothing to do with our internal cloud ). We need to use the cloud controller IP for the request and the port it is bound (by default 8773 if you have not changed it). This will look like this (you need to run it from inside the actual instance): Read the rest of this entry »

s3sync is a great tool to synchronize local data with Amazon S3 for backups, or whatever other reasons you might want to put your data on S3. It is very simple to install (gem install s3sync) and use (s3sync -v -s -r –progress <source_dir> s3_bucket:<dir>); it runs very well and it can be easily scripted to do regular backups or even synchronize live data with S3. The only problem I found while using s3sync was that it can be very slow when uploading a lot of data (millions of files) to S3; this because the process is slow but also because it runs a single file at a time, and it doesn’t do several uploads in parallel. I would have loved for s3sync to do this out of the box, but unfortunately it doesn’t, but for my particular need I was able to do this by running more s3sync commands a the same time. It will not apply to your data (unless it is structured the same way as here; very unlikely), but it might give an idea on how you could do this your own data if it is structured in a feasible way.

Read the rest of this entry »

A while ago one of the major problems people faced to use Amazon EC2 into production environments was the dynamic state of the instances IPs. Every time one instance was started it was getting a new, dynamic IP. This has been addressed with the introduction of Amazon Elastic IP Addresses, but even when using this, the private IPs are still dynamic and most of the time people will want to communicate between several instances on the private allocated IPs and not on the public ones. This article will show how you can easily automate the process to update DNS hostnames for your EC2 instances, by adding to the AMI’s the logic for this. I will use for this a master DNS server running bind9, but this can be adapted to any other DNS server. Read the rest of this entry »

Between 20-24 April 2009 I attended to CloudSlam09, the 1st annual virtual conference on cloud computing. This was a global event, using the webex technology, covering latest trends and innovations in the world of cloud computing. Being a virtual event it allowed me to easily attend and listen to several keynotes and talks from the comfort of my home/work office. Obviously this is a huge advantage as not so many people can leave their jobs for 1 week to go at such events. This was a great idea for such an event and a perfect match for the topic.

Besides the obvious advantages of a virtual event, I would like to outline also some disadvantages: besides a rather poor sound quality for some speakers, the biggest minus for me was the inability to focus on a single talk uninterrupted. This because, being actually at work I had to deal with all the usual stuff that I have to do normally (the ones called emergencies). Being on a real event would imply other type of ‘noise’ but anyway .

Read the rest of this entry »

Amazon just announced “reserved instances”, guaranteeing uptime and a price reduction if customers commit to Amazon’s cloud solution for a year or more.

“We’ve learned that some of our customers have needs which aren’t addressed by the spot pricing model. For example, some of them were looking for even lower prices, and were willing to make a commitment ahead of time in order to achieve this. Also, quite a few customers actually told us something even more interesting: they were interested in using EC2 but needed to make sure that we would have a substantial number of instances available to them at any time in order for them to use EC2 in a DR (Disaster Recovery) scenario. In a scenario like this, you can’t simply hope that your facility has sufficient capacity to accommodate your spot needs; you need to secure a firm resource commitment ahead of time.”

Read the rest of this entry »

Amazon just extended its Amazon Elastic Compute Cloud (Amazon EC2) to Europe. S3 has bee available for Europe for about one year, and now we can also run EC2 instances in Europe to achieve lower latencies with EU locations. The new EU region called eu-west-1.ec2.amazonaws.com is completely isolated from the US region (us-east-1.ec2.amazonaws.com) to achieve the greatest possible failure independence and stability. This means that resources found on the US region (like ami’s, ssh keypairs, etc.) are not available to the EU region and will need to be created (or copied/mirrored) here also.

“With today’s launch, you can take advantage of the latest features for Amazon EC2 including multiple Availability Zones, Elastic IP addresses, and Amazon Elastic Block Store (Amazon EBS). In the near future, Amazon EC2 will also add support for Windows Server and SQL Server in the EU which is a new feature that was recently introduced on Amazon EC2 in the U.S.”

Read the rest of this entry »

Cloud computing management software RightScale secured $13 million in second round venture funding (after first round $4.5 million back in April).

Michael Crandell, CEO of RightScale, commented: “Over the next five years, spending on IT cloud services is expected to grow almost threefold, reaching $42 billion by 2012, according to industry research firm IDC. Specializing from its inception exclusively on cloud computing technology, RightScale is well positioned to maintain its leadership position as this industry matures. The company has two years of real-world cloud computing experience and thousands of live deployments. Over 200,000 servers have been successfully launched using the RightScale cloud management platform.”

Full Press Release: http://www.rightscale.com/news_events/press_releases/2008/08december2008.php

Today Amazon announced the public beta of Amazon CloudFront, their AWS service for content delivery. This is the service that many users of Amazon S3 (Simple Storage Service) have been waiting for a long time. Even if S3 was never a ‘real’ CDN (content delivery network) it was used by many sites to serve static content. The main limitation of this approach was that it had no geographical awareness as content delivery networks usually have; the fact that S3 is highly scalable and well priced made this solution acceptable on S3.

CloudFront is the answer to all users’ requests about using S3 as a CDN, delivering the content using a global network of 14 edge locations. CloudFront uses S3 to store the original file, and caches copies of the content close to end users locations, lowering latency when they download the objects.

Read the rest of this entry »

Rackspace announced today that it has agreed to buy Slicehost and Jungle Disk for a total of $18 million. This move is supposed to improve its Mosso cloud offerings to better compete with Amazon Web Services.

Slicehost is a leader in Xen-based virtual machine hosting with more than 15,000 “slices” online today. Jungle Disk offers reliable cloud storage solutions that allow users to easily share an unlimited amount of cloud storage between multiple users through a secure, mountable network drive and automatic backup.

http://ir.rackspace.com/phoenix.zhtml?c=221673&p=irol-newsArticle&ID=1215812&highlight=