Keeping up with the traffic | Auto Scaling

4 min readAug 24, 2022

Let’s assume that you added more features to your website. It gathers information from user, writes to database, shows users various data. It’s not just website anymore. It’s a web-application now. People started noticing your application and it became viral. Tons of people interacted with it. Your VM is getting busier than ever. VM’s CPU and memory is all time high. The website is getting less responsive by minute. You don’t need a crystal ball to see the disaster.

In this situation, you can

start a new VM
do some network changes so that some of the traffic goes to our new VM

This operation is also called scale out.You successfully alleviated pressure from the first VM for some time. Pretty soon you notice an uptick in CPU and memory usage again and you need to scale up again. Finally, the traffic on your application normalize and reduces slowly. Now, you need to get rid of the VMs which is also known as scale in.

Next day in the morning, traffic surges and fades away in the same way. This is why, you may want to automate scale up/down.

Requirements of the automated scale up and down process:

Keep track of key performance indicators such as CPU and memory
Create VM with necessary services installed already or find a way to install them after VM creation
Make network changes to route traffic

Furthermore, There was an power outage at the data center where you have created your VM. To withstand this kind of scenario, it is recommended to create VM in a distant data center. So we can add 4th requirement is as following

Ability to create VM in different data center(s)

AWS has a concept of Auto Scaling. As name suggests, Auto Scaling helps to scale up (to a certain threshold) or down without manual intervention.

Requirement 1: keep track of a key performance indicator

Keeeping-track-of-your-vm shows how to use CloudWatch to monitor a VM. CloudWatch integrates well with other AWS services including Auto Scaling.

Requirement 2: Create VMs with necessary services installed already or find a way to install them afterwards

You can start with base image, install necessary software on it using Userdata field
Or you can spin up a VM, install necessary software and then take a snapshot of it. Some important snapshots are known as “golden image”.
Using configuration management tool — As name implies, Configuration management tools (such as Ansible, Chef, Puppet) helps to configure a machine. Using Configuration Management tools, one can create a file which would outline the steps to install and configure a software. You can use the same file and install the software on 10/100 or 1000s of machines. Many companies provide these files to their users.

Requirement 3: Network changes to route traffic

If you create a VM, it would have public IP-address A. If you reboot it after some time, it would have completely different public IP-address. We can only link a static IP address with DNS.

In distribute-the-incoming-traffic post, I mentioned load balancers at high level. AutoScaling and Load Balancer can communicate with each other.

Requirement 4: Ability to create VM in different data center(s)

AutoScaling can create VMs in data centers of your choice. Having VMs across different data center means that your app can withstand data center outages.

On-demand instances are costly which is why AutoScaling can also provision spot instances. That’s cherry on top of the cake.

Get-started-with-ec2-auto-scaling page outlines how to setup auto scaling. To auto scale VM, you need to tell

what to create? = information related to VM
where to create? = which Virtual Private Cloud and data centers
how many to create? = this is called group sizing. Here you would need to specify maximum, minimum and desired number of VMs
when to scale? = (option A) you can configure target tracking feature which would keep an eye on a metric and scale accordingly. This is called dynamic scaling. (option B) other scaling options such as scheduled scaling and predictive scaling

Other things such as:

want to integrate with Load balancer ?
want to get notified ?

With the combination of DNS — Load Balancer and Auto Scaling, a website may look like following

In this post, I covered

Scale out/in = this terms are different than scale up or down. Scale up means increasing disk size, memory or CPU of an instance whereas scale out means creating extra instances
Auto Scaling
CloudWatch — to monitor resources
Golden Image —

Keeping up with the traffic | Auto Scaling

Written by Jigar Rathod