Develop a CI/CD Pipeline to Roll Out Containerized Microservices Using Rolling Deployment

Udacity - Cloud DevOps Engineer Nanodegree

Capstone Project: Develop a CI/CD Pipeline to Roll Out Containerized Microservices Using Rolling Deployment

Capstone Project Overview

As a capstone project, the directions are rather more open-ended than they were in the previous projects in the program. You will also be able to make some of your own choices in this capstone, for the type of deployment you implement, which services you will use, and the nature of the application you develop.

You will develop a CI/CD pipeline for micro services applications with either blue/green deployment or rolling deployment. You will also develop your Continuous Integration steps as you see fit, but must at least include typographical checking (aka “linting”). To make your project stand out, you may also choose to implement other checks such as security scanning, performance testing, integration testing, etc.!

Project Tasks

Once you have completed your Continuous Integration you will set up Continuous Deployment, which will include:

• Pushing the built Docker container(s) to the Docker repository (you can use AWS ECR, create your own custom Registry within your cluster, or another 3rd party Docker repository) ; and
• Deploying these Docker container(s) to a small Kubernetes cluster. For your Kubernetes cluster you can either use AWS Kubernetes as a Service, or build your own Kubernetes cluster. To deploy your Kubernetes cluster, use either Ansible or Cloudformation. Preferably, run these from within Jenkins as an independent pipeline.

You can find a detailed project rubric, here.

Project Details

Pipeline Deployment

Initializing Stage:

Lint HTML Stage:

Build Docker Image Stage:

Push to Docker Hub Stage:

Deploy to Cluster Stage:

Take Down Stage:

Linting Stage Verification

HTML Linting Check Catches Invalid HTML Tag:

HTML Linting Check Passes Corrected HTML Tag:

Deployment Type

Rolling Deployment

• Deployment Logs:

+ kubectl get services
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP                                                              PORT(S)          AGE
capstone-project-elb   LoadBalancer   10.100.77.85   ac84e215e577d4eb9a6fa4ac3d7708a6-229427463.us-east-1.elb.amazonaws.com   8080:32092/TCP   4h25m
kubernetes             ClusterIP      10.100.0.1     <none>                                                                   443/TCP          4h43m
+ kubectl get pods -o wide
NAME                                             READY   STATUS    RESTARTS   AGE     IP               NODE                             NOMINATED NODE   READINESS GATES
cloud-devops-capstone-project-7ff8ff4c6f-2k8h4   1/1     Running   0          4h25m   192.168.15.201   ip-192-168-31-109.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-58v9w   1/1     Running   0          4h25m   192.168.48.46    ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-6b5qj   1/1     Running   0          4h25m   192.168.50.176   ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-9sbr4   1/1     Running   0          4h25m   192.168.31.21    ip-192-168-31-109.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-c8tp6   1/1     Running   0          4h25m   192.168.56.227   ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-jz5z9   1/1     Running   0          4h25m   192.168.59.12    ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-mr6mf   1/1     Running   0          4h25m   192.168.11.249   ip-192-168-31-109.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-qqvnr   1/1     Running   0          4h25m   192.168.3.228    ip-192-168-31-109.ec2.internal   <none>           <none>
+ kubectl apply --filename=k8-deployment-config.yml
deployment.apps/cloud-devops-capstone-project configured
service/capstone-project-elb unchanged
+ kubectl get services
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP                                                              PORT(S)          AGE
capstone-project-elb   LoadBalancer   10.100.77.85   ac84e215e577d4eb9a6fa4ac3d7708a6-229427463.us-east-1.elb.amazonaws.com   8080:32092/TCP   4h25m
kubernetes             ClusterIP      10.100.0.1     <none>                                                                   443/TCP          4h43m
+ kubectl get pods -o wide
NAME                                             READY   STATUS              RESTARTS   AGE     IP               NODE                             NOMINATED NODE   READINESS GATES
cloud-devops-capstone-project-54cb4855b6-5v967   1/1     Running             0          2s      192.168.19.119   ip-192-168-31-109.ec2.internal   <none>           <none>
cloud-devops-capstone-project-54cb4855b6-7j5bt   0/1     ContainerCreating   0          2s      <none>           ip-192-168-31-109.ec2.internal   <none>           <none>
cloud-devops-capstone-project-54cb4855b6-gjdkh   1/1     Running             0          2s      192.168.63.222   ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-54cb4855b6-hgdql   1/1     Running             0          2s      192.168.60.141   ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-2k8h4   1/1     Running             0          4h25m   192.168.15.201   ip-192-168-31-109.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-58v9w   1/1     Running             0          4h25m   192.168.48.46    ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-6b5qj   1/1     Running             0          4h25m   192.168.50.176   ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-c8tp6   0/1     Terminating         0          4h25m   192.168.56.227   ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-jz5z9   1/1     Running             0          4h25m   192.168.59.12    ip-192-168-32-247.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-mr6mf   1/1     Running             0          4h25m   192.168.11.249   ip-192-168-31-109.ec2.internal   <none>           <none>
cloud-devops-capstone-project-7ff8ff4c6f-qqvnr   1/1     Running             0          4h25m   192.168.3.228    ip-192-168-31-109.ec2.internal   <none>           <none>

Deployment Services and Pod Details Before and After Deployment to Cluster

EKS Cluster

CloudFormation Stack Info - EKS Cluster

CloudFormation Events - EKS Cluster

CloudFormation Resources - EKS Cluster

Docker application:

Nginx "Hello World, my name is (student name)" application.

• I used an nginx image, specifically the alpine tag - nginx:stable-alpine:
  • This image is based on the popular Alpine Linux project, available in the alpine official image. Alpine Linux is much smaller than most distribution base images (approx. 5MB), and thus leads to much slimmer images in general.

Docker Hub Repository

Application

Requirements

• Git is installed.
• Python 3 is installed.
• Pip is installed.
• Jenkins is installed and configured correctly.
• Docker is installed and configured.
• Docker Hub Account for image repository.
• The Kubernetes command-line tool, kubectl is installed and configured.
• The AWS Command Line Interface is installed.
• The aws-iam-authenticator is installed.
• eksctl - the official CLI tool for Amazon EKS is installed.
• The Tidy HTML linter is installed.

Configuration

• Software dependencies are installed.
• Set up IAM credentials in AWS.
• Launch EC2 Instance for Jenkins box and configure.
• Install Jenkins and configure.
• Verify Blue Ocean plugin is installed in Jenkins.
• Verify Docker plugin is installed in Jenkins.
• Verify Pipeline: AWS Steps plugin is installed in Jenkins.
• Set up a GitHub Repository.
• Add GitHub credentials to Jenkins.
• Add Docker Hub credentials to Jenkins.
• Add AWS credentials to Jenkins.
• Configure AWS CLI credentials.
• Use the aws-iam-authenticator script.
• Create the Amazon EKS cluster.
  • This was accomplished by running the k8_cluster_initializer.sh script.
• Update the Amazon EKS cluster once its creation is complete.
  • This was accomplished by running the k8_cluster_constructor.sh script.
• Add Pipeline.

References

• Using a Jenkinsfile
• End-to-End Multibranch Pipeline Project Creation
• nginx documentation
• nginx image:
• nginx:stable-alpine tag
• Docker Installation on Ubuntu
• Docker Pipeline
• Pushing docker images to Docker Hub
• Installing kubectl
• Overview of kubectl
• Deployments
• Kubernetes API Reference Docs for Deployment v1 apps
• Interacting with running pods
• eksctl - The official CLI for Amazon EKS
• Amazon EKS Workshop
• Getting started with eksctl
• Installing eksctl
• Installing aws-iam-authenticator
• Managing users or IAM roles for your cluster
• How To Authenticate to AWS with the Pipeline AWS Plugin
• Jenkins Pipeline Step Plugin for AWS
• Amazon EKS cluster endpoint access control
• Create a kubeconfig for Amazon EKS
• Creating an Amazon EKS cluster
• Kubernetes LoadBalancer
• Troubleshoot Service Load Balancers for Amazon EKS