GitHub – Pxxmie/Terraform_Jenkins_S3

github.com

GitHub – Pxxmie/jenkins-pipeline-deploy-to-s3

github.com

In this project, I will be using Terraform to orchestrate the setup of a Jenkins environment on AWS EC2, create an S3 bucket, and establish a robust CI/CD pipeline to efficiently deploy a static website on AWS S3.

Step 1: Set Up Terraform

Ensure that Terraform is installed and configured correctly on your local machine. If you haven’t installed Terraform yet, follow these steps:

Install Terraform: https://developer.hashicorp.com/terraform/install

and,

Verify Installation:

After installation, ensure that Terraform is properly installed and accessible from your terminal by running the following command:

terraform --version 

Step 2: Create a Jenkins Installation Script

This step involves creating a bash script to automate the installation of Jenkins on an EC2 instance. Below are detailed instructions for creating the script:

Create a new directory;

mkdir terraform-tf
cd terraform-tf

Create a Jenkins installation script;

nano jenkins.sh

#!/bin/bash

# Update the system packages
sudo yum update -y

# Download and add the Jenkins repository
sudo wget -O /etc/yum.repos.d/jenkins.repo 
    https://pkg.jenkins.io/redhat-stable/jenkins.repo

# Import the Jenkins repository key
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key

# Upgrade the system packages
sudo yum upgrade -y

# Install Java 11 (Amazon Corretto)
sudo dnf install java-11-amazon-corretto -y

# Install Jenkins
sudo yum install jenkins -y

# Enable Jenkins to start on system boot
sudo systemctl enable jenkins

# Start the Jenkins service
sudo systemctl start jenkins

# then install git
sudo yum install git -y

#then install terraform
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo
sudo yum -y install terraform

#finally install kubectl
sudo curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.23.6/bin/linux/amd64/kubectl
sudo chmod +x ./kubectl
sudo mkdir -p $HOME/bin && sudo cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin

Step 3: Configure AWS Provider

The AWS provider configuration specifies the AWS region and credentials for Terraform to use when provisioning resources.

Create provider.tf;

# Configure the AWS provider
provider "aws" {
    region  = "eu-west-1"    # Specify the AWS region
    profile = "prismika"     # Optional: Use a named profile for authentication
}

Step 4: Define VPC Configuration

In this step, you define the configuration for the Virtual Private Cloud (VPC) where our Jenkins server on the EC2 instance will be deployed.

Create a vpc.tf;

# Define the AWS VPC
resource "aws_vpc" "jenkins-vpc" {
    # Specify the CIDR block for the VPC
    cidr_block = "10.0.0.0/16"
    
    # Enable DNS support for the VPC
    enable_dns_support = true
    
    # Enable DNS hostnames for the VPC
    enable_dns_hostnames = true

    # Add tags to the VPC for identification
    tags = {
        Name = "jenkins-vpc"
    }
}

Define a public subnet in vpc.tf;

# Define the public subnet within the VPC
resource "aws_subnet" "public-jenkins-subnet" {
    # Associate the public subnet with the VPC using its ID
    vpc_id            = aws_vpc.jenkins-vpc.id
    
    # Specify the CIDR block for the public subnet
    cidr_block        = "10.0.1.0/24"
    
    # Specify the availability zone for the public subnet
    availability_zone = "eu-west-1a"

   # Ensure that instances launched in this subnet will have public IP addresses
    map_public_ip_on_launch = true

    # Add tags to the public subnet for identification
    tags = {
        Name = "public-jenkins-subnet"
    }
}

Step 5: Create Internet Gateway

To enable internet access for the VPC, you need to create an internet gateway.

Create network.tf;

# Create an internet gateway for Jenkins VPC
resource "aws_internet_gateway" "jenkins-igw" {
    vpc_id = aws_vpc.jenkins-vpc.id    # Associate the internet gateway with the Jenkins VPC
    tags = {
        Name = "jenkins-igw"           # Assign a name tag to the internet gateway
    }
}

Create a custom route table on network.tf;

# Define a route table for the public subnet
resource "aws_route_table" "public-jenkins-rt" {
    vpc_id = aws_vpc.jenkins-vpc.id

route {
        cidr_block = "0.0.0.0/0"
        gateway_id = aws_internet_gateway.jenkins-igw.id
    }
    tags = {
        Name = "public-jenkins-rt"
    }
}

on the same file associate route table and subnet;

# Associate the public subnet with the public route table
resource "aws_route_table_association" "public-subnet-association" {
    subnet_id      = aws_subnet.public-jenkins-subnet.id
    route_table_id = aws_route_table.public-jenkins-rt.id
}

Finally, create a security group;

# Create a new security group
resource "aws_security_group" "jenkins-sg" {
  description = "Allows port SSH, HTTP, and Jenkins traffic"
  vpc_id      = aws_vpc.jenkins-vpc.id

# Define security group rules for Jenkins EC2 instance
ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]  # Allow SSH access from anywhere
}
ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]  # Allow HTTP access from anywhere
}
ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]  # Allow Jenkins access from anywhere
}
egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"            # Allow all outbound traffic
    cidr_blocks = ["0.0.0.0/0"]  
}
tags = {
    Name = "Jenkins-sg"           # Assign a name tag to the security group
}

Step 6: Create a Terraform Configuration File

Create a file named main.tf; this file contains the configuration for provisioning the EC2 instance, installing Jenkins, and creating an S3 bucket.

# Define an AWS EC2 instance
resource "aws_instance" "jenkinsec2" {
  ami                    = "ami-062a49a8152e4c031"
  instance_type          = "t2.micro"
  key_name               = "prismikaec2"
  subnet_id              = aws_subnet.public-jenkins-subnet.id
  security_groups        = [aws_security_group.jenkins-sg.id]
  associate_public_ip_address = true
  user_data              = file("jenkins.sh")  # Include user data script for Jenkins configuration

  tags = {
    Name = "jenkinsec2"
  }
}

# Define an AWS S3 bucket for hosting the static website
resource "aws_s3_bucket" "my-static-web-bucket" {
  bucket = "jenkinstf-ec2-static-bucket"  # Replace with your desired bucket name
}

# Configure static website hosting for the S3 bucket
resource "aws_s3_bucket_website_configuration" "my-static-web-bucket" {
  bucket = aws_s3_bucket.my-static-web-bucket.id
  index_document {
    suffix = "index.html"
  }
  error_document {
    key = "error.html"
  }
}

# Apply ownership controls for the S3 bucket
resource "aws_s3_bucket_ownership_controls" "example" {
  bucket = aws_s3_bucket.my-static-web-bucket.id

  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

# Configure public access block settings for the S3 bucket
resource "aws_s3_bucket_public_access_block" "block_public_access" {
  bucket = aws_s3_bucket.my-static-web-bucket.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

Step 7: Initialise, Plan & Apply the configuration

After creating the Terraform configuration file, you’ll need to initialise Terraform, plan the changes, and apply the configuration to provision the resources.

terraform init
terraform plan
terraform apply 

Step 5: Validate AWS Resources

EC2 Instance:

Security group:

VPC:

S3 Bucket:

Step 6: Access Jenkins

To access the Jenkins server on your EC2 instance, follow these steps:

Copy the public DNS of your EC2 Instance:

Open a web browser and paste the public DNS of your EC2 followed by:8080

<http://your-ec2-public-dns:8080>

• A prompt will appear requesting an admin password to unlock Jenkins.

Go back onto your EC2 instance and click Connect on the top right.

You can retrieve the password to unlock jenkins by running the following command;

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

• Copy the admin password from the file and paste it into the Jenkins web interface.
• Follow the on-screen instructions to complete the setup wizard. You can install suggested plugins or manually select plugins based on your requirements.

CI/CD Pipeline: Deploying a Static Website to AWS S3 with Jenkins

After setting up our Jenkins environment and S3 bucket, the next step is to build a CI/CD pipeline to host our static website.

Plugins required:

• S3 publisher plugin
• Pipeline: AWS steps

Important:

Make sure Maven is installed, you can do this by going to manage jenkins, scrolling down to Maven, entering a name and clicking on install automatically. Apply and save.

Step 1: Configure Jenkins Credentials for GitHub and AWS Access

• Click on your username in the top-right corner of the page and select credentials from the dropdown menu.
• On the Credentials page, click on global to manage global credentials then click on add credentials on the top left.

Below, I’ve configured my GitHub credentials by providing my username and password, and for AWS, I’ve used secret text to input both the access key and secret access key separately.

Step 2: Create an HTML file

Set up a Public Folder:

In your repository, create a folder named public to store your HTML file. Inside the public folder, create a new file with the extension index.html. This file will contain the HTML code for your website.

<!DOCTYPE html>
<html>
<head>
<style>
body {
  margin-left: 0px;
}

.center_div {
  margin-left: auto;
  margin-right: auto;
  width: 100%;
  background-color: #FFF5FD;
  text-align: center;
  padding: 10px;
}

</style>
</head>
<body>

<div>
  <h1>Hello World!</h1>
  <p>This page is hosted on a static website.</p>
</div>

</body>
</html>

Next, create another HTML file named error.html;

<!DOCTYPE html>
<html>
<head>
<style>
body {
  margin-left: 0px;
}

.center_div {
  margin-left: auto;
  margin-right: auto;
  width: 100%;
  background-color: #FFF5FD;
  text-align: center;
  padding: 10px;
}

</style>
</head>
<body>

<div>
  <h1>Error page !!! </h1>
  <p>This page is not working.</p>
</div>

</body>
</html>

Step 3: Create a Jenkinsfile

In the same repository, create a Jenkinsfile to define your CI/CD pipeline. This file will contain the instructions for Jenkins on how to build, test, and deploy your static website.

#!/usr/bin/env groovy

// Define the Jenkins pipeline
pipeline {
    agent any // Run this pipeline on any available agent

    // Define environment variables for AWS credentials and region
    environment {
        AWS_ACCESS_KEY_ID = credentials('AWS_ACCESS_KEY_ID') // Access Key ID from Jenkins credentials
        AWS_SECRET_ACCESS_KEY = credentials('AWS_SECRET_ACCESS_KEY') // Secret Access Key from Jenkins credentials
        AWS_DEFAULT_REGION = "eu-west-1" // AWS region to use
    }

    // Define stages for the pipeline
    stages {
        // Stage for deploying files to S3 bucket
        stage('Deploy to S3') {
            steps {
                // Execute shell commands to copy HTML files to S3 bucket
                script {
                    sh "aws s3 cp public/index.html s3://jenkinstf-ec2-static-bucket/"
                }
            }
        }

        // Stage for setting bucket policy
        stage('Set Bucket Policy') {
            steps {
                // Execute shell command to set bucket policy using policy.json file
                script {
                    sh "aws s3api put-bucket-policy --bucket jenkinstf-ec2-static-bucket --policy file://policy.json"
                }
            }
        }
    }
}

Step 4: Create a .json file

Create a new file in the same repository and name it policy.json. In this file, define the bucket policy for your S3 bucket.

{
    "Version": "2012-10-17",  // Specifies the version 
    "Statement": [  
        {
            "Sid": "PublicReadGetObject",  // Unique identifier for the statement
            "Effect": "Allow",  // Specify the effect (Allow/Deny)
            "Principal": "*",  // Allow access to anyone
            "Action": "s3:GetObject",  // Define the action allowed (GetObject)
            "Resource": "arn:aws:s3:::

Finally, push your repository to GitHub so that Jenkins can access it to configure the build.

Step 5: Create a Pipeline job in Jenkins

Navigate to your Jenkins dashboard, and click on new item to create a new job. Give your job a name and select Pipeline.

Scroll down to Pipeline and select Pipeline Script from SCM under Definition followed by selecting GIT under SCM.

Now go onto your GitHub and copy your repository URL.

Paste your repo URL under Repository URL followed by selecting your GitHub credentials that were configured earlier. I have specified my branch as main. Apply and save your job.

Step 6: Building your pipeline

After creating your pipeline job, navigate to the job’s dashboard and click on Build Now to trigger the build.

Once the build is triggered, you can monitor the progress of your job in the Stage View. This view provides a visual representation of the different stages defined in your pipeline.

If any stages fail, you can review the logs on the console output to identify the cause and make any necessary adjustments to your pipeline configuration.

Step 7: Verify the website on S3 Bucket

On your AWS Console go to Amazon S3 service and find the S3 bucket that you specified in your Jenkins pipeline for hosting the static website.

Inside the bucket, ensure that both of your HTML files are present.

Select properties and scroll down to Static website hosting, click on the provided URL http://jenkinstf-ec2-static-bucket.s3-website-eu-west-1.amazonaws.com to access your static website.

you can also access error.html at http://jenkinstf-ec2-static-bucket.s3-website-eu-west-1.amazonaws.com/error.html

Step 8: Clean Up with Terraform Destroy

It’s important to tidy up the resources created by Terraform when you’re done using them. To do this, simply run the following command in your terminal;

terraform destroy