HowTo update DNS hostnames automatically for your Amazon EC2 instances

A while ago one of the major problems people faced to use Amazon EC2 into production environments was the dynamic state of the instances IPs. Every time one instance was started it was getting a new, dynamic IP. This has been addressed with the introduction of Amazon Elastic IP Addresses, but even when using this, the private IPs are still dynamic and most of the time people will want to communicate between several instances on the private allocated IPs and not on the public ones. This article will show how you can easily automate the process to update DNS hostnames for your EC2 instances, by adding to the AMI’s the logic for this. I will use for this a master DNS server running bind9, but this can be adapted to any other DNS server.

How to get the needed information (IPs, hostnames, etc.)

Amazon api provides us all the needed information. Any EC2 instance can get a lot of information about itself just by querying a web server using a REST-like API. Here is how we can get all the available metadata items:
curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
ancestor-ami-ids
block-device-mapping/
hostname
instance-action
instance-id
instance-type
kernel-id
local-hostname
local-ipv4
placement/
public-hostname
public-ipv4
public-keys/
ramdisk-id
reservation-id
security-groups

We have a direct interest in the public-ipv4 and local-ipv4 variables, but as we can see Amazon is providing many other useful information that can be used inside the instance for various purposes. For more details checkout the amazon docs.

Public and private IPs

This means that for getting the public and private ips we only have to make two calls like this:
curl http://169.254.169.254/latest/meta-data/local-ipv4and
curl http://169.254.169.254/latest/meta-data/public-ipv4

Hostnames

We need a way to identify from inside the instance what hostname this should be configured. There are probably several ways to do this, but the most common is to specify this when the instance is started using the –user-data option of the ec2-run-instances command (or the short form -d). This will pass the custom data and make it available to the instance.

You will probably want to customize this based on your needs. Myself I assumed that I will use the same domain for all instances and I need to pass only the hostname. Since I don’t need any other parameters to the machine I can just do this:
ec2-run-instances <AMI> -d "myhostname" ...other params...
If you use more user data, then you will probably use it like “hostname=myhostname <other_variables>”. (in this case you will need to update the script bellow like this: HOSTNAME=`echo $USER_DATA | cut -f 1 -d , | cut -f 2 -d =`)

Now making a http request like this will give us the hostname this instance should have:
curl http://169.254.169.254/latest/user-data

DNS Server configuration

Now that we have available inside the EC2 instance all the needed information, we need a way to allow the instance to update the DNS server. As I mentioned before, I will use for this a bind9 server, but this can by any other DNS server that allows this action to be scripted somehow (either using some api calls, or any way to use dynamic DNS update). For bind9 we will use the nsupdate utility to update the DNS server securely using the dnssec key mechanism.

Personally I don’t use the full domain for this, but delegate two subdomains (to the same nameservers) like this:
; ec2 zones:
ec2.domain.com.      NS      ns1.domain.com.
ec2.domain.com.      NS      ns2.domain.com.
ec2-int.domain.com.  NS      ns1.domain.com.
ec2-int.domain.com.  NS      ns2.domain.com.
and allow update access only to those zones. Of course if you prefer that you can give direct access to your full domain zone.

Generate a key using the dnssec-keygen utility like this:
dnssec-keygen -a HMAC-MD5 -b 512 -n USER user.domain.com.
and this will create two files like this:
Kuser.domain.com.+157+47950.key
Kuser.domain.com.+157+47950.private

Using the information from the public key add to your dns server configuration the key:

key user.domain.com. {
algorithm HMAC-MD5;
secret "xAw7F/axmVSxsZ+V4LAZnkeYObjOaJjbVKf21Zl4WhxtRHdlhqWSeCdd fIVR6MhC8LSQoim7NfkWD2j7WT5AHw==";
};

where secret is the value from the public key, that in my example looks like this:

cat Kuser.domain.com.+157+47950.key
user.domain.com. IN KEY 0 3 157 xAw7F/axmVSxsZ+V4LAZnkeYObjOaJjbVKf21Zl4WhxtRHdlhqWSeCdd fIVR6MhC8LSQoim7NfkWD2j7WT5AHw==

Finally we need to allow update access for the key:

zone "ec2.domain.com"
{
type master;
file "/etc/bind/zone/ec2.domain.com";
allow-update { key user.domain.com.; };
allow-query { any; };
};

zone "ec2-int.domain.com"
{
type master;
file "/etc/bind/zone/ec2-int.domain.com";
allow-update { key user.domain.com.; };
allow-query { any; };
};

Bind will need to be restarted after making these changes.

Using nsupdate to update the hostname

Next we will need to upload the key we created on the EC2 image (later we will save it inside the AMI once all runs well) and test to see if it is working properly.
cat<<EOF | /usr/bin/nsupdate -k Kuser.domain.com.+157+47950.private -v
server ns1.domain.com
zone ec2.domain.com
update delete test.ec2.domain.com A
update add test.ec2.domain.com 60 A <some_IP>
show
send
EOF

If this is working properly we can move on and put all this toghether in a script that will be running at the instance start time. If not, go back and see in your dns server logs if there are any issues why this is not working.

Finally automation :-)

Now we just have to put all the pieces together and using a simple script like this will do the job:
ec2-hostname.sh:

#!/bin/bash

#you will need to have the key available in the instance in the same dir as this script
DNS_KEY=Kuser.domain.com.+157+47950.private
DOMAIN=domain.com

USER_DATA=`/usr/bin/curl -s http://169.254.169.254/latest/user-data`
HOSTNAME=`echo $USER_DATA`
#set also the hostname to the running instance
hostname $HOSTNAME.$DOMAIN

PUBIP=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/public-ipv4`
cat<<EOF | /usr/bin/nsupdate -k $DNS_KEY -v
server ns1.$DOMAIN
zone ec2.$DOMAIN
update delete $HOSTNAME.ec2.$DOMAIN A
update add $HOSTNAME.ec2.$DOMAIN 60 A $PUBIP
send
EOF

LOCIP=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/local-ipv4`
cat<<EOF | /usr/bin/nsupdate -k $DNS_KEY -v
server ns1.$DOMAIN
zone ec2-int.$DOMAIN
update delete $HOSTNAME.ec2-int.$DOMAIN A
update add $HOSTNAME.ec2-int.$DOMAIN 60 A $LOCIP
send
EOF

You will probably want to run this at boot time either from rc.local or creating an initscript for it. I put all these EC2 related stuff under /usr/local/ec2 and in this case I just call it from rc.local with a line like this:
/usr/local/ec2/ ; sh ec2-hostname.sh

If all runs as you wanted you will probably want to save your AMI to include the script that will automatically update the dns hostname at instance boot time.

Hopefully you found this article interesting and it will be a starting point to create your own dns update script based on your needs.

Tags: , , , , ,

Related Posts:

Posted in Cloud Computing, Linux, Tips & Tricks | by - Marius -
You can leave a response, or trackback from your own site.

13 Responses to “HowTo update DNS hostnames automatically for your Amazon EC2 instances”

Leave a Reply

« Prev:
Next: »

Marius on Twitter

Recent Posts

Archives