Infrastructure as Code — CloudFormation - Part 1

6 min readJul 30, 2022

This post is a continuation of Intro to Infrastructure as Code (IaC)— https://jigarr.medium.com/save-your-private-cloud-setup-intro-to-infrastructure-as-code-4fa89a23ecec. CloudFormation tool works exclusively with Amazon Web Services (AWS).

Here are few important questions one would encounter while using IaC.

How to define resources ?

All of the resources such as Virtual machine, security group, elastic IP address have few things in common. Each resource has a name, type and resource specific properties. List of resources and various properties you can use is listed at https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html

Multiple resources can be defined in a JSON/YAML file which would be uploaded to AWS’s storage service (Simple Storage Service — S3). JSON/YAML file can be seen as a blue-print or template of your infrastructure. CloudFormation would read the template and create resources as per the template file.

What if I want to create a volume and mount it on a virtual machine?

In this case, you would define 2 resources (with logical id Ec2Instance and NewVolume) in the template file as following.

Resources:
  Ec2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      KeyName: 'JigarMacbook'
      ImageId: "ami-0cff7528ff583bf9a"
      AvailabilityZone: "us-east-1a"
    Volumes:
    - Device: "/dev/sdf"
      VolumeId: !Ref NewVolume
  NewVolume:
    Type: AWS::EC2::Volume
    Properties:
      Size: 1
      Encrypted: true
      AvailabilityZone: "us-east-1a"
      Tags:
        - Key: CreatedUsing
          Value: CloudFormation_jrathod
      DeletionPolicy: Snapshot

Notice that we used !Ref NewVolume to reference the volume. Ref is a function supported by CloudFormation. In case of YAML, functions are prefixed with !.NewVolume and Ec2Instance is logical id. All the resources in a template can be visualized as a part of a single unit called Stack.

If you try to create a Stack using the YAML file outlined above, CloudFormation would create a volume and then it would an error:“The key pair ‘JigarMacbook’ does not exist”. At this point, it would try to rollback by deleting the resources it created so far. You can configure CloudFormation to keep the resources it created so far. You can also define, how a resource should be deleted using DeletionPolicy. In this case, I asked it to take a snapshot of the volume.

It’s a good practice to minimize fixed (hard-coded) values. I can customize above template so that when you would run it, it would ask you to input key-pair of your choice. This can be done by defining key as a parameter. The updated template would look as following:

Parameters:
  KeyName:
    Description: KeyPair to SSH into VM
    Type: 'AWS::EC2::KeyPair::KeyName'Resources:
  Ec2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      KeyName: !Ref KeyName
      ImageId: "ami-0cff7528ff583bf9a"
      AvailabilityZone: "us-east-1a"
    Volumes:
    - Device: "/dev/sdf"
      VolumeId: !Ref NewVolume
  NewVolume:
    Type: AWS::EC2::Volume
    Properties:
      Size: 1
      Encrypted: true
      AvailabilityZone: "us-east-1a"
      Tags:
        - Key: CreatedUsing
          Value: CloudFormation_jrathod
      DeletionPolicy: Snapshot

When you would try to run above now, you would see following in the console.

Let’s print the IP address of the VM that we created using CloudFormation. In order to do so, we need to add Outputs in the YAML file

Parameters:
  KeyName:
    Description: KeyPair to SSH into VM
    Type: 'AWS::EC2::KeyPair::KeyName'Resources:
  Ec2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      KeyName: !Ref KeyName
      ImageId: "ami-0cff7528ff583bf9a"
      AvailabilityZone: "us-east-1a"
    Volumes:
    - Device: "/dev/sdf"
      VolumeId: !Ref NewVolume
  NewVolume:
    Type: AWS::EC2::Volume
    Properties:
      Size: 1
      Encrypted: true
      AvailabilityZone: "us-east-1a"
      Tags:
        - Key: CreatedUsing
          Value: CloudFormation_jrathodOutputs:
  VM1ipAddress:
    Description: Public IP address of our EC2
    Value: !GetAtt Ec2Instance.PublicIp

Notice that I used !GetAtt function to fetch public IP address. Also, Output of one stack can be imported in another.

We created resources in us-east-1a availability zone. What if I want to create resources in a different Availability Zone such as us-west-1?

Part 1 — support us-west-1

The ami-0cff7528ff583bf9a only exists in us-east-1 region. If we want to create a VM machine in another region, we need to have relevant imageId. Instead of replacing imageId and AvailabilityZone from our template, we can create a map by adding mappings section to our template

Mappings:
  RegionMap:
    us-east-1:
      image: ami-0cff7528ff583bf9a
    us-west-1:
      image: ami-02d1e544b84bf7502

Part 2 — get region

We create a parameter so that user would select the region and create resources in it. Let’s take it a step further by detecting the region automatically. If a user is creating a Stack in us-east-1 then create resources in that region. This can be achieved by using pseudo parameter AWS::Region.

Part 3 — lookup map to find relevant imageId

Lookup AWS::Region in RegionMap and get the value of image key. This can be achieved by using !FindInMap function. Putting it together:

ImageId: !FindInMap [RegionMap, !Ref 'AWS::Region', image]

Part 4 — update volume related properties

AvailabilityZone (AZ) is required property of a volume. Users are logged into a region, not into an availability zone. However, one does not need to specify AZ while creating a virtual machine. We can solve this by taking following steps:

Create VM — Remove AZ related line from the template
find out in which AZ the VM was created

       AvailabilityZone: !GetAtt Ec2Instance.AvailabilityZone

Create Volume in the same AZ

Mappings:
  RegionMap:
    us-east-1:
      image: ami-0cff7528ff583bf9a
    us-west-1:
      image: ami-02d1e544b84bf7502Parameters:
  KeyName:
    Description: KeyPair to SSH into VM
    Type: 'AWS::EC2::KeyPair::KeyName'Resources:
  Ec2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      KeyName: !Ref KeyName
      ImageId: !FindInMap [RegionMap, !Ref 'AWS::Region', image]
    Volumes:
    - Device: "/dev/sdf"
      VolumeId: !Ref NewVolume
  NewVolume:
    Type: AWS::EC2::Volume
    Properties:
      Size: 1
      Encrypted: true
      AvailabilityZone: !GetAtt Ec2Instance.AvailabilityZone
      Tags:
        - Key: CreatedUsing
          Value: CloudFormation_jrathod

If you run above template, you would get an error“Circular dependency between resources: [Ec2Instance, NewVolume]”. To create a VM, we need a volume and to create a volume, we need to find AZ in which VM will be created.

This can be resolved if we can

Create VM without reference to Volume
Create Volume in the same AZ as VM
Attach Volume with the VM — using VolumeAttachement resource

Mappings:
  RegionMap:
    us-east-1:
      image: ami-0cff7528ff583bf9a
    us-west-1:
      image: ami-02d1e544b84bf7502Parameters:
  KeyName:
    Description: KeyPair to SSH into VM
    Type: 'AWS::EC2::KeyPair::KeyName'Resources:
  Ec2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      KeyName: !Ref KeyName
      ImageId: !FindInMap [RegionMap, !Ref 'AWS::Region', image]

  NewVolume:
    Type: AWS::EC2::Volume
    Properties:
      Size: 1
      Encrypted: true
      AvailabilityZone: !GetAtt Ec2Instance.AvailabilityZone
      Tags:
        - Key: CreatedUsing
          Value: CloudFormation_jrathod  MountEc2Instance:
    Type: AWS::EC2::VolumeAttachment
    Properties:
      InstanceId: !Ref Ec2Instance
      VolumeId: !Ref NewVolume
      Device: /dev/sdf

What would happen if a user is creating a stack in ap-south-1 region?

CloudFormation would report an error “Template error: Unable to get mapping for RegionMap::ap-south1::image”

What would happen if I create another stack with the same template file?

Resources will be created per stack. So you would end up with multiple resources.

What if someone changes the size of the volume directly in the console?

CloudFormation can detect changes to the stack resources. It can figure out how resources have drifted from it’s original configurations. One can simply View drift results via console.

Making changes to the Stack

You can make changes to the template and then create a change set by uploading the updated template. CloudFormation would detect the necessary changes and update the infrastructure accordingly.

The items that I covered in this post is just the tip of the iceberg. I will cover few more concepts at a higher level in part 2 of this post.

In this blog, I covered few important CloudFormation specific terminologies

Template — YAML/JSON file outlining various resources
Stack — Combination of resources
Function — has a prefix ! (For example, !Ref)
Parameters — Input needed to run a CloudFormation template
Psuedo Parameters — AWS::Region
Outputs
Mappings
Rollback configurations
Deletion policy — how to delete a resource
Change-set
Drift results — Difference between template and the real setup

Reference

AWS CLI document — https://docs.aws.amazon.com/cli/latest/reference/

AWS CloudFormation template — cfn-whatis-concepts.html

CloudFormation masterclass 2015 — https://youtu.be/6R44BADNJA8

CloudFormation under the hood 2013 — https://youtu.be/ZhGMaw67Yu0

Deep Dive of CloudFormation — https://youtu.be/KXUsyApAI3Y

CloudFormation — https://youtu.be/1h-GPXQrLZw

Infrastructure as Code — CloudFormation - Part 1

Written by Jigar Rathod