AWS, Terraform, NGINX, Ansible, Docker, Jenkins

AWS, Terraform, NGINX, Ansible, Docker, Jenkins

Disclosure: scottyfullstack.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to amazon.com.

Scotty Parlor

March 14, 2021

Read Time 34 min

                





This will be the first time I have written a tutorial directly from a suggestion in our slack channel. I haven't done suggestions up to this point as to try and make sure the channel continues in the right direction to build upon specific skills to get into DevOps.

This suggestion fits perfectly with the flow of what I have been doing and I certainly hope it helps all who read (particularly the requester)!

I was sent a list of requirements that an employer expected to see in a live demo. Here is the list:

Automate provisioning and configuration of NGINX server in Docker container with SSL certificate in AWS cloud using Jenkins, Ansible, Terraform.
Assumptions:
1.  The available free version of Linux
2.  The latest available version of NGINX for Linux
3.  EC2 instance size is t2.micro (free tier)
4.  Must create and use free tier of private AWS account
5.  The SSL certificate is self-signed
6.  The AWS Security Group is default
Acceptance Criteria:
1.  Post-provisioning report demonstrates evidence of
a.  No provisioning errors
b.  Sequence of provisioning steps
c.  Up and running instance of NGINX
d.  Listening on ports 80 and 443
e.  Response on ports 80 and 443
f.  content of active instance configuration file
2.  Run automated pipeline 3 times in a row and receive the same result (fully remove provisioned EC2 with components after every deployment, automation is optional for removal)
This one was actually fun to work on. So let's try to meet each of these criteria.

Assumptions to complete this tutorial:

  1. You have an AWS account set up and have IAM profiles on your local.
  2. You have Jenkins installed and know how to basically use it
  3. You have Github (and CLI) with ssh keys set up.
  4. You have terraform installed
  5. You have Docker Installed
  6. Basic knowledge of EC2 and security groups
  7. A key pair already set up to utilize. In this example, mine is devops.pem


Our project directory will look like this by the end:

.
├── ansible
│   └── static-site
│   └── site.yaml
├── Jenkinsfile
├── nginx
│   ├── Dockerfile
│   └── static-site
│   ├── index.css
│   ├── index.html
│   └── nginx.conf
└── terraform
└── static-site
├── main.tf
├── terraform.tfstate
├── terraform.tfstate.backup
└── variables.tf



Prior to beginning, as always, setup your github repo and get ready to showcase your effort.

gh repo create scottyfullstack/nginx-jenkins-ansible-terraform

Then inside that git repo run the following to get the proper ignore file to ignore the terraform junk:

wget -O .gitignore https://raw.githubusercontent.com/github/gitignore/master/Terraform.gitignore

Part 1: NGINX, OpenSSL, and the static site all bundled into Docker


The requirement listed here is for an NGINX Docker container to be deployed onto a linux EC2 host with a self signed SSL cert.

For this, we can show off just a little bit and embellish the static hosted site across NGINX.

Let's create a directory for our nginx, static-site, and all the required files.

mkdir nginx
cd nginx
touch Dockerfile
mkdir static-site
cd static site
touch index.html index.css nginx.conf

Let's start by adding the html and css quick. Of course, feel free to do whatever you want to this.

index.html

<!DOCTYPE html>

<html>
<head>
    <title>Nginx EC2</title>
    <link href="index.css" rel="stylesheet">
</head>
<body>
    <main class="main">
        <h1>Hi, I'm Scotty</h1>
    </main>
</body>
</html>


index.css
body {
    margin:0;
    font-family: Arial, Helvetica, sans-serif;
    background: linear-gradient(to right, #5c258d, #4389a2);
    color:white;
}

.main {
    width:100%;
    height:100vh;
    display:flex;
    justify-content: center;
    align-items: center;
}

Next, we can add the standard nginx config, only with some modifications (I have trimmed out the options and added the SSL server with the locations of where our certs will be in the next step):

Note: you can find the basic config here

nginx.conf

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;

    keepalive_timeout  65;

    server {
        listen       80;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;

        location = /50x.html {
            root   html;
        }
    }

    server {

        listen 443 http2 ssl;
        listen [::]:443 http2 ssl;

        server_name localhost;

        ssl_certificate /etc/nginx/certs/nginx-selfsigned.crt;
        ssl_certificate_key /etc/nginx/private/nginx-selfsigned.key;

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        
        location = /50x.html {
            root   html;
        }
    }

}


Now, save those and open up the Dockerfile one level up:

Here we are taking the base and lastest Nginx image from docker and then making the dirs for our certs and copying our static files over into /etc/.

Then we are running the OpenSSL command to generate a self signed cert with our information. In this case, you will get a security warning when accessing the ec2 from the browser. Go ahead and replace my information with your information in that RUN command.

Dockerfile
FROM nginx

RUN mkdir /etc/nginx/private /etc/nginx/certs
COPY static-site/index.html /etc/nginx/html/
COPY static-site/index.css /etc/nginx/html/
COPY static-site/nginx.conf /etc/nginx/nginx.conf

RUN openssl req -x509 -nodes -days 365 \
-subj "/C=US/ST=FL/O=Parlor Design, LLC/CN=scottyfullstack.com" \
-newkey rsa:2048 -keyout /etc/nginx/private/nginx-selfsigned.key \
-out /etc/nginx/certs/nginx-selfsigned.crt;

finally in this step, we can build and push the docker image to your own docker repo:

docker build . -t scottyfullstack/nginx-static --no-cache
docker push scottyfullstack/nginx-static



Part 2: Terraform


Before we tackle the Terraform, make sure you have a security group set up for SSH and TCP. In the requirements they say "default", but that is usually used for VPC internal communication of services. You can decide how you want to handle security, but I've created a secondary SG with the specific requirements.



The Terraform piece is almost an exact copy from DevOps 01 so if you need a refresher, check that post out.

back out in our projects root level folder, create the terraform dir and then its child static-site:

mkdir terraform
cd terraform
mkdir static-site
cd static-site
touch main.tf variables.tf

main.tf
Note that you have to fill in your SG id's

terraform {
  required_version = "0.12.20"
}

provider "aws" {
    region=var.region
    profile=var.profile
}

resource "aws_instance" "site" {
    ami = "ami-042e8287309f5df03"
    instance_type="t2.micro"
    key_name="devops"
    vpc_security_group_ids = ["sg-f72319ab","sg-0586fa0aa906bfb79"]
        tags = {
            Name = var.name
            group = var.group
        }
}

variables.tf

variable "profile" {
    description = "The profile used to auth to AWS"
}

variable "region" {
    description= "The region our instance will be in (i.e. us-east-1)"
}

variable "name" {
    description= "The name of the instance we are creating"
}

variable "group" {
    description= "the name of the group we will be using for Ansible purposes"
}

And that is all for Terraform. EASY.



Part 3: Ansible and Dynamic Inventory


If you haven't checked out my other post and video Button Click Environment I will be relying heavily on that tutorial to save time and space.

Before you forget, make sure to cp your aws credentials AND EC2 key to the jenkins user if that's the route you are going to take:

sudo cp ~/.aws/credentials /var/lib/jenkins/.aws/
sudo cp ~/.ssh/devops.pem /var/lib/jenkins/.ssh/
sudo chown jenkins:jenkins /var/lib/jenkins/.aws/credentials
sudo chown jenkins:jenkins /var/lib/jenkins/.ssh/devops.pem

We want to have ansible run and utilize a Dynamic Inventory to account for EC2 instances coming and going.

For this we need the following directory/files:

/etc/ansible/aws_ec2.yaml
/etc/ansible/group_vars/tag_group_web.yaml
/etc/ansible/ansible.cfg
./ansible/static-site/site.yaml

so let's start with the /etc/ folder. Recall that the boto_profile is your credential for JENKINS in this case.

/var/lib/jenkins/.aws/credentials

/etc/ansible/aws_ec2.yaml

plugin: amazon.aws.aws_ec2
boto_profile: scottyfullstack
regions:
  - us-east-1
strict: False
keyed_groups:
  - prefix: tag
    key: 'tags'
compose:
  ansible_host: ip_address

/etc/ansible/group_vars/tag_group_web.yaml

ansible_ssh_private_key_file: /var/lib/jenkins/.ssh/devops.pem
ansible_user: ubuntu

/etc/ansible/ansible.cfg

[defaults]
host_key_checking = False

[ssh_connection]
retries=3

and finally our ansible playbook in our directory (note the image at the end should be YOUR docker image:

./ansible/static-site/site.yaml

---
    - name: Provision Web Servers
      hosts: tag_group_web
      tasks:
    
        - name: Install pip3
          apt:
            update_cache: yes
            name: python3-pip
          become: yes
    
        - name: Install python docker sdk
          shell: |
            pip3 install docker
          become: yes
    
        - name: Install docker
          apt:
            name: docker.io
          become: yes
    
        - name: Start Docker
          shell: |
            systemctl start docker
            systemctl enable docker
          become: yes

        - name: Run image
          shell: docker run --name hello -dit -p 80:80 -p 443:443 scottyfullstack/nginx-static
          become: yes

And that's all for Ansible.



Part 4: Jenkins Pipeline


Now for the grand finale, the Jenkins Pipeline.

For the jenkins piece, we will be running jenkins locally and pulling from Git. So push it all up there now.

git add .
git commit -m "Devops Project"
git push origin master

Fire up jenkins and log in.

Create a new job and select "Pipeline" (call it something intuitive)

Now, I have a Jenkinsfile in my repo that is arbitrary. I keep it there to show off, as well as make edits easily when I need to. We are ultimately going to past it into Jenkins, so its up to you if you want to include it or not.

Take a moment to review it. Note the following:

  1. It has params at the top
  2. it checks my repo using Jenkins credentials that I supplied in my instance. These are SSH credentials (private key is in jenkins, public key is in Github). I have covered this in other videos.
  3. The Terraform stage has an if/else statement for apply or destroy
  4. Ansible has a retry block in case the connection doesn't work on first try (to avoid the error penalty). Also, note that it uses the -i to reference our Dynamic inventory.

Jenkinsfile

pipeline {
  agent any
  parameters {
    choice(
        name: 'Action',
        choices: "apply\ndestroy",
        description: 'Apply or Destroy the instance' )
  }
  stages {

    stage('Checkout') {
        steps {
            git branch: 'master', credentialsId: 'Github', url: 'git@github.com:ScottyFullStack/nginx-jenkins-ansible-terraform.git'
        }
    }

    stage('Terraform') {
      steps {
        script {
          if (params.Action == "apply") {
            sh 'terraform init terraform/static-site'
            sh 'terraform apply -var "name=hello" -var "group=web" -var "region=us-east-1" -var "profile=scottyfullstack" --auto-approve terraform/static-site'
          } 
          else {
            sh 'terraform destroy -var "name=hello" -var "group=web" -var "region=us-east-1" -var "profile=scottyfullstack" --auto-approve terraform/static-site'
          }
        }
      }
    }

    stage('Ansible') {
      steps {
        retry(count: 5) {
          sh 'ansible-playbook -i /etc/ansible/aws_ec2.yaml ansible/static-site/site.yaml'
        }
      }
    }
  }
}

Scroll down to the bottom where it says "Pipeline" and paste in the Jenkinsfile script. Save it and reload.

Try Building it with parameters and see how it goes!

You should be able to see each stage complete:



When you are done standing it up you can navigate to your EC2 url and verify it works.

You should see the ssl warning when using https




You can verify your SSL info as well by clicking the "Not Secure" at the top left.

Click through it and see your static site:



When you are done, You can head back to Jenkins, select "Destroy" and run it! Your verify your instance is terminated and run it for them as many times as they want!



That concludes this tutorial. I hope you have enjoyed it!