Using AWS CLI to create a high Availability Architecture

In this article I’ll be sharing my experience of creating a high availability architechture using AWS CLI. The architechture includes :

1. Launching an AWS EC2 instance and configuring it as a web server.
2. Making the document root persistent using EBS block device.
3. Using S3 to store static objects(eg: images).
4. Setting up a Content Delivery Network using CloudFront and using the origin domain as an S3 bucket.
5. Finally using the url generated by CloudFront to display the image in our website for security and latency.

What is CloudFront?

Let’s start off by looking at what a CDN is and why we use one? CDN stands for Content Delivery Network. It is a very large distribution of caching servers that are located around the world. It contains content that are stored in your origin servers and routes the viewers to the best location so that they can view the content that is stored in the cache. The content can be static (content that does not change)or dynamic (content that does change) in nature. CDN adds to the scalability and performance factor of applications.

CloudFront is Amazon’s global content delivery network with massive capacity and scale. It is optimised for performance and scalability. Security features are also built in and you can configure them for optimal service. The user is in control of the service and can make changes on the fly. It includes real time reporting so that you can monitor the performance and make changes to the application or the way the CDN interfaces with your application. It has been optimised for static and dynamic objects and video delivery.

Having this initial knowledge of CloudFront we are good to go now…

1. Launching an AWS EC2 instance and configuring it as a web server

Launching an EC2 instance using AWS CLI is very simple, refer to previous blog on How to launch an AWS EC2 instance using AWS CLI?

After successfully launching an ec2 instance now we’ll configure it as a web server. We are going to use a server software provided by Apache known as Apache HTTPD. To install this software in our instance run the command yum install httpd .

2. Making the document root persistent using EBS block device

After installing httpd, next step is to put the html code of our website in the document root. Httpd bydefault picks the code file from the location /var/www/html directory. However in case of system failure due to any reason, all our code and sensitive data is at high risk, therefore one option is to keep our code in an EBS block device.

For this we have to first create an EBS volume and then attach this volume to our instance. In our case I’ll be creating an EBS volume of 5 GiB using the AWS CLI command: aws ec2 create-volume --availability-zone us-east-1e --volume-type gp2 --size 5

and then attaching it ec2 instance using the command: aws ec2 attach-volume --instance-id i-03eed7ce0af839519 --volume-id vol-0e6eb8a390f9cf688 --device /dev/sdf

To use this block device we have to first perform 3 operations viz. create partition, format the partition and then mount it.

• create partition using fdisk /dev/xvdf where xvdf is the device name. Press m to know about all the options which we can use. Press n to create a new partition and then select primary partition by typing p . Press w to save this partition.

Use fdisk -l to know about all the volumes attached to our instance.

• format the partition using: mkfs.ext4 /dev/xvdf1

• and then finally mount this partition to our document root i.e. /var/www/html using the command: mount /dev/xvdf1 /var/www/html

Now we can store our code in the document root. For this first navigate to the /var/www/html directory and here we’ll create a simple website in the index.html file which contains code for displaying a welcome message and an image.

Finally we can start our webserver service using the command systemctl start httpd . Verify it using the status command( systemctl status httpd)

3. Using S3 to store static objects

We have used EBS block device to store our code and data, but there’s still a small risk of data availability and durability when it comes to EBS. AWS provides a better option than EBS to store such static objects by its service named S3 (Simple Storage Service).

Amazon S3 or Amazon Simple Storage Service is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface. Amazon S3 uses the same scalable storage infrastructure that Amazon.com uses to run its global e-commerce network. Amazon S3 can be employed to store any type of object which allows for uses like storage for Internet applications, backup and recovery, disaster recovery, data archives, data lakes for analytics, and hybrid cloud storage.

Although Amazon Web Services (AWS) does not publicly provide the details of S3’s technical design, Amazon S3 manages data with an object storage architecture which aims to provide scalability, high availability, and low latency with 99.999999999% durability and between 99.95% to 99.99% availability (though there is no service-level agreement for durability).

The basic storage units of Amazon S3 are objects which are organized into buckets. Each object is identified by a unique, user-assigned key. Buckets can be managed using either the console provided by Amazon S3, programmatically using the AWS SDK, or with the Amazon S3 REST application programming interface (API).

To create a bucket using AWS CLI method use the command: aws s3api create-bucket --acl public-read --bucket durvesh999 , where durvesh999 is the bucket name which should be unique in the entire region in which you’re working. We are allowing public read access to our bucket using the --acl option.

After which we’ll upload an image file (referred to as Object in terms of S3). Use the following command to put an object in the bucket and give public read access to our object:

aws s3api put-object --bucket durvesh999 --key img3.png --body C:\Users\Durvesh\Desktop\ARTH\img3.png

aws s3api put-object-acl --bucket durvesh999 --key img3.png --acl public-read

4. Setting up a Content Delivery Network using CloudFront and using the origin domain as an S3 bucket

Let’s take a look at some of the components of CloudFront.

Distributions

Distributions is the instantiation of CloudFront. It acts as a pointer to the original content that you are hosting either in an AWS or custom origin. The origins need to be specified in the distribution so that CloudFront knows where to get the content when a request comes in and we don’t have the content stored in the cache.

Origins

Origins describes the Amazon S3 bucket or HTTP server or could be a EC2 instance from which CloudFront gets the content. At least one origin should be created. To maintain security and make sure that your origin is delivering content to CloudFront there are a couple of things that you can do. You can use a OAI (Origin Access Identity) for S3 so that it will restrict access for S3 bucket to just CloudFront. Any other request not coming from CloudFront will be denied. You can use a Signed URL, such that CloudFront can use that to access the origin and origin will respond only if the content of the signed URL is valid. You can also use an Origin Custom Header that goes back from CloudFront to your origin and based on the value of the header the origin will search for the header and if it does not exist or the value is incorrect the request will be denied.

CloudFront Regional Edge Caches are regional caches deployed around the world and it acts as a caching tier between the edge location and the origin. Earlier, when there were no Regional Edge Caches, the CDN edge locations had to go back to the origin when they had no content in them. With Regional Edge Caches, the edge locations would go to the origins only if the Regional Edge Caches did not have the content in them. Regional Edge Caches lies between the CDN edge locations and the origins and helps reduce the load on the origin, allows you to scale the CDN without having to scale the origin and does not incur any additional costs. Regional Edge Caches have larger caches than edge location so objects will remain longer in them.

To configure CloudFront first you have to create a CloudFront distribution so that CloudFront knows where to deliver the content from. You specify origin servers ( S3 buckets or HTTP servers ) to store your objects (files). You upload the files which can be web pages, images and media files to your origin servers. Then you create the distributions so that CloudFront knows which origin servers to get the files from. You can also specify whether you want to log all requests and whether you want to enable the distribution as soon as it is created. CloudFront assigns a domain name to your distribution which you can see in the console. CloudFront will send the configuration of the distribution to all of its edge locations.

To create a CloudFront Distribution using AWS CLI use the command:

aws cloudfront create-distribution --origin-domain-name durvesh999.s3.amazonaws.com --default-root-object img3.png

5. Finally using the url generated by CloudFront to display the image in our website for security and latency

Use the above generated Domain Name for the image to be shown in our website code:

Let’s check if every thing is done correctly as per our requirement.

And here it is ! Everything is perfectly done !!

THANKS FOR READING!