This chapter is from the book
This chapter is from the book
This chapter is from the book
Application Layer
The application layer is where you'll probably spend most of your time because it is the heart and soul of any SaaS system. This is where your code translates data and requests into actions, changing, manipulating, and returning data based on inputs from users, or other systems. This is the only layer that you have to actually maintain and scale, and even then, some cloud providers offer you unique solutions to handle that automatically for you. In Google AppEngine, this is handled automatically for you. In Amazon Web Services, this can be handled semi-automatically for you by using Auto-Scaling Groups, for which you can set rules on when to start and stop instances based on load averages or other metrics.
Your application layer is built on top of a base image that you created and may also contain scripts that tell it to update or add more code to that running instance. It should be designed to be as modular as possible and enable you to launch new modules without impacting the old ones. This layer should be behind a proxy system that hides how many actual modules are in existence. Amazon enables you to do this by providing a simple service known as Elastic Load Balancing, or ELB.
Using Elastic Load Balancing
Amazon's Elastic Load Balancing, or ELB, can be used simply and cheaply to proxy all requests to your modules based on their Instance ID. ELB is even smart enough to proxy only to systems that are actually live and processing, so you don't have to worry about server failures causing long-term service disruptions. ELB can be set up to proxy HTTP or standard TCP ports. This is simple to accomplish using code and can even be done on the actual instance as it starts, so it can register itself when it's ready to accept connections. This, combined with Auto-Scaling Groups, can quickly and easily scale your applications seamlessly in a matter of minutes without any human interaction. If, however, you want more control over your applications, you can just use ELB without Auto-Scaling Groups and launch new modules manually.
Creating and managing ELBs is quite easy to accomplish using boto and the elbadmin command-line tool that I created, which comes with boto. Detailed usage of this tool can be found by running it on the command line with no arguments:
% elbadmin
Usage: elbadmin [options] [command]
Commands:
list|ls List all Elastic Load Balancers
delete <name> Delete ELB <name>
get <name> Get all instances associated
with <name>
create <name> Create an ELB
add <name> <instance> Add <instance> in ELB <name>
remove|rm <name> <instance> Remove <instance> from ELB
<name>
enable|en <name> <zone> Enable Zone <zone> for ELB
<name>
disable <name> <zone> Disable Zone <zone> for ELB
<name>
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-z ZONES, --zone=ZONES
Operate on zone
-l LISTENERS, --listener=LISTENERS
Specify Listener in,out,proto
The first thing to do when starting out is to create a new ELB. This can be done simply as shown here:
% elbadmin -l 80,80,http -z us-east-1a create test Name: test DNS Name: test-68924542.us-east-1.elb.amazonaws.com Listeners --------- IN OUT PROTO 80 80 HTTP Zones --------- us-east-1a Instances ---------
You must pass at least one listener and one zone as arguments to create the instance. Each zone takes the same distribution of requests, so if you don't have the same amount of servers in each zone, the requests will be distributed unevenly. For anything other than just standard HTTP, use the tcp protocol instead of http. Note the DNS Name returned by this command, which can also be retrieved by using the elbadmin get command. This command can also be used at a later time to retrieve all the zones and instances being proxied to by this specific ELB. The DNS Name can be pointed to by a CNAME in your own domain name. This must be a CNAME and not a standard A record because the domain name may point to multiple IP addresses, and those IP addresses may change over time.
Recently, Amazon also released support for adding SSL termination to an ELB by means of the HTTPS protocol. You can find instructions for how to do this on Amazon's web page. At the time of this writing, boto does not support this, so you need to use the command-line tools provided by Amazon to set this up. The most typical use for this will be to proxy port 80 to port 443 using HTTPS. Check with the boto home page for updates on how to do this using the elbadmin command-line script.
Adding Servers to the Load Balancer
After you have your ELB created, it's easy to add a new instance to route your incoming requests to. This can be done using the elbadmin add command:
% elbadmin add test i-2308974
This instance must be in an enabled zone for requests to be proxied. You can add instances that are not in an enabled zone, but requests are not proxied until you enable it. This can be used for debugging purposes because you can disable a whole zone of instances if you suspect a problem in that zone. Amazon does offer a service level agreement (SLA), ensuring that it will have 99% availability, but this is not limited to a single zone, thus at any given time, three of the four zones may be down. (Although this has never happened.)
It's generally considered a good idea to use at least two different zones in the event one of them fails. This enables you the greatest flexibility because you can balance out your requests and even take down a single instance at a time without effecting the service. From a developer's perspective, this is the most ideal situation you could ever have because you can literally do upgrades in a matter of minutes without having almost any impact to your customers by upgrading a single server at a time, taking it out of the load balancer while you perform the upgrade.
Although ELB can usually detect and stop proxying requests quickly when an instance fails, it's generally a good idea to remove an instance from the balancer before stopping it. If you're intentionally replacing an instance, you should first verify that the new instance is up and ready, add it to the load balancer, remove the old instance, and then kill it. This can be done with the following three commands provided in boto package:
% elbadmin add test i-2308974 % elbadmin rm test i-0983123 % kill_instance i-0983123
The last command actually terminates the instance, so be sure there's nothing on there you need to save, such as log files, before running this command. After each of these elbadmin commands, the full status of that load balancer is printed, so be sure before running the next command that the previous command succeeded. If a failure is reported, it's most likely because of an invalid instance ID, so be sure you're copying the instance IDs exactly. One useful tool for this process is the list_instances command, also provided in the boto package:
% list_instances ID Zone Groups Hostname ------------------------------------------------------------------ i-69c3e401 us-east-1a Wordpress ..compute-1.amazonaws.com i-e4675a8c us-east-1c default ..compute-1.amazonaws.com i-e6675a8e us-east-1d default ..compute-1.amazonaws.com i-1a665b72 us-east-1a default ..compute-1.amazonaws.com
This command prints out the instance IDs, Zone, Security Groups, and public hostname of all instances currently running in your account, sorted ascending by start date. The last instances launched will be at the bottom of this list, so be sure to get the right instance when you're adding the newest one to your ELB. The combination of these powerful yet simple tools makes it easy to manage your instances and ELB by hand.
Although the load balancer is cheap (about 2.5 cents per hour plus bandwidth usage), it's not free. After you finish with your load balancer, remove it with the following command:
% elbadmin delete test
Automatically Registering an Instance with a Load Balancer
If you use a boto pyami instance, you can easily tell when an instance is finished loading by checking for the email sent to the address you specify in the Notification section of the configuration metadata passed in at startup. An example of a configuration section using gmail as the smtp server is shown here:
[Notification] smtp_host = smtp.gmail.com smtp_port = 587 smtp_tls = True smtp_user = my-sending-user@gmail.com smtp_pass = MY_PASSWORD smtp_from = my-sending-user@gmail.com smtp_to = my-recipient@gmail.com
Assuming there were no error messages, your instance should be up and fully functional. If you want the instance to automatically register itself when it's finished loading, add an installer to your queue at the end of your other installers. Ensure that this is done after all your other installers finish so that you add only the instance if it's safe. A simple installer can be created like the one here for ubuntu:
from boto.pyami.installers.ubuntu.installer import Installer
import boto
class ELBRegister(Installer):
"""Register this instance with a specific ELB"""
def install(self):
"""Register with the ELB"""
# code here to verify that you're
# successfully installed and running
elb_name = boto.config.get("ELB", "name")
elb = boto.connect_elb()
b = ebl.get_all_load_balancers(elb_name)
if len(b) <1:
raise Exception, "No Load balancer found"
b = b[0]
b.register_instances([boto.config.get_instance ("instance_id")])
def main(self):
self.install()
This requires you to set your configuration file on boot to contain a section called ELB with one value name that contains the name of the balancer to register to. You could also easily adapt this installer to use multiple balancers if that's what you need. Although this installer will be called only if all the other installers before it succeed, it's still a good idea to test anything important before actually registering yourself with your balancer.