Teacher: Vincent Gauthier
Assistant: Gatien Roujanski
- Clone this repository
- Each challenge must be validated with a professor and then committed and pushed to your own github repository.
- Before starting a given step, present the sketch of your infrastructure to a professor.
- We strongly recommend that you use Alpine OS as the base operating system for your docker image.
On your own computer install the following software:
- Docker Desktop
- Conda
- IDE (VSCode, etc)
- Build a one page Flask application which contains the following elements:
- Your name
- Your project name
- Version of your website (i.e. V1)
- The server hostname
- The current date
- Build a docker container which contains your Flask web page
- Test your application
- Send your docker container on DockerHub
- Draw a schema of your systems (ex. draw.io)
- Show the system
- Show the container IP address
- Show the container ports
Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. It has no database abstraction layer, form validation, or any other components where pre-existing third-party libraries provide common functions. However, Flask supports extensions that can add application features as if they were implemented in Flask itself.
A minimal Flask application looks something like this:
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello_world():
return "<p>Hello, World!</p>"Now you can run the web serveur you just created
$ flask --app hello run
* Serving Flask app 'hello'
* Running on http://127.0.0.1:5000 (Press CTRL+C to quit)See the quickstart guide for more information
- Getting started with Flask
- Containerize an application
- Christian Lempa - Learning Docker // Build Container Images
- Find a already made mongodb docker container (for instance the official mongodb docker image)
- Use docker compose to deploy your mongodb database
- Test your application and add record in your database manually with the mongod command (see the install requirement for mongodb
- Update the schema of your infrastructure (ex. draw.io)
- Show the system
- Show the containers IP address or hostname
- Show the container ports
MongoDB is classified as a NoSQL database program, MongoDB uses JSON-like documents. MongoDB provides high availability with replica sets. A replica set consists of two or more copies of the data. Each replica-set member may act in the role of primary or secondary replica at any time. All writes and reads are done on the primary replica by default. Secondary replicas maintain a copy of the data of the primary using built-in replication. When a primary replica fails, the replica set automatically conducts an election process to determine which secondary should become the primary. Secondaries can optionally serve read operations, but that data is only eventually consistent by default. It's easier to create a master/slave database cluster when using nosql.
- Update your Flask application and add the following items:
- Your name
- Your project name
- Version of your website (i.e. V1)
- The server hostname
- The current date
- Flask application should connect to a mongodb database each time a request is served :
- It connects to the mongodb database through pymongo
- For each request, it will record in the mongodb database:
- The IP address of the client
- The current date
- Your flask application should display the last 10 records of the database
- Update Flask app version displayed on the page to V2
- Finaly deploy your services using a docker compose file with the following elements:
- Docker service for your website (your flask application)
- Docker service for your mongodb database
- Network to connect the previous services
- Send your docker container on DockerHub with the correct tags.
- Update the schema of your infrastructure (ex. draw.io)
- Show the system
- Show the container IP address
- Show the container ports
- Docker compose gettig started
- Configure the default network
- How to get the last N records in mongodb?
Add a NGINX load balancer to your Docker compose file. Update your docker compose file with the following elements:
- 2 Flask application
- Deploy 1 Flask app without database connections
- Deploy 1 Flask app with database connections
- 1 NGINX load balancer which balances the load between the two web server
- 1 Mongodb database
- Network
- Update the schema of your infrastructure (ex. draw.io)
- Show the system
- Show the container IP address and the hostname of each container
- Show the container ports
- Ask the professor to get access to the Kubernetes server: register your Telecom SudParis email ([email protected]) as a valid google address or provide valid gmail address ([email protected]).
- Install gcloud cli
- Install kubectl
- You should have access to the cluster dashboard at the following address: https://console.cloud.google.com/kubernetes
- Login with gcloud
$ gcloud auth login
...- Select net-4255 as the default project
$ gcloud config set project net-4255 - Install the kubeconfig file on your computer
$ gcloud container clusters get-credentials net4255-gke --location=us-central1-a --project=net-4255-404310After this command you should have kubeconfig.yml file on your laptop. This file will enable you to get access to the kubernetes cluster.
$ ls $HOME/.kube/
kubeconfig.ymlExport kubeconfig file
$ export KUBECONFIG=$HOME/.kube/kubeconfig.ymlNow, you should be able to connect to the kuberntest cluster. Check the following command in order to check the connection.
$ kubectl cluster-info
...You should see a namespace with your name !!! with the following command:
$ kubectl get namespace- Follow the Kubernetes Tutorial
Create your first deployment in command line with the kubectl command:
- Create a deployment for the webnodb container in your own namespace
- Without any replicat
- Without any service
- Check that your deployment is successful in the google cloud console and with the command line:
$ kubectl get deployments -o wide
...
$ kubectl get pods -o wide
...- Test the pod web server id correctly running with port-forwarding
$ kubectl port-forward pods/xxxx your pod name xxxx :xxx pod port xxx --namespace=xxxx your namespace xxxx
Forwarding from 127.0.0.1:54127 -> 5000Now you can connect with your browser to http://127.0.0.1:54127 to access the webnodb website.
- Deploying your first app on Kubernetes
- Utiliser le Port Forwarding pour accéder à des applications dans un cluster
- Create a Deployment file for your container webnodb (the one without database)
- Be careful, create your Deployment and the ClusterIP service in your on namespace !!!!
- Show on a new schema how a request is served from the service to the pods:
- The schema should explain which port is used at each step and what IP address is used by each component (nodes, pods, services)
- Request the following resources per pod:
- cpu resource: 1/10 CPU per pod
- memory: 100 Mo per pod
- Limit your pod resources as the following:
- cpu resourse: 1/5 of CPU per pod
- memory: 200 Mo per pod
- Connect to the cluster through a proxy with the following command or use port-forwarding to test your application:
$ kubectl proxy
Starting to serve on 127.0.0.1:8001 Now should be able to access to the webnodb web page at the following url http://127.0.0.1:8001/api/v1/namespaces/__your_namespace_name__/services/__your_service_name__/proxy/
- Update the schema of your infrastructure (ex. draw.io)
- Show the system
- Show the container IP address and the hostname of each container
- Show the container ports
- Kubernetes YAML File Explained - Deployment and Service
- Accessing services running on the cluster
- Manually constructing apiserver proxy URLs
- Resource Management for Pods and Containers
Challenge 8: Deploy on the Kubernetes cluster your website (webdb) and the respective mongodb database (2pts)
- Deploy the webdb web service with 3 replica and its related service (NodePort)
- Deploy the mongodb database and its related service (ClusterIP)
- Connect the web service to the database using kubernetes DNS hostname
- Explain the difference between a NodePort Service and a ClusterIP service
- Validate your deployment (webdb, mongodb) by using port-forwarding
- Request the following resources per pod:
- cpu resource: 1/10 CPU per pod
- memory: 100 Mo per pod
- Limit your pod resources as the following:
- cpu resourse: 1/5 of CPU per pod
- memory: 200 Mo per pod
- Update the schema of your infrastructure (ex. draw.io)
- Show the system
- Show the container IP address and the hostname of each container
- Show the container ports
- Create a Ingress in order to expose your web application (webnodb and webdb)
- GKE has some custom-made feature in order to deploy Ingress, be sure to read carefully to Ingress documentation of GKE
- Don't forget to deploy your ingress in your own namespace !!!
- The kubenetes cluster has a public IP address with the hostname: "net4255.luxbulb.org" create a service that redirects http traffic of the following url to your respective deployment:
- url 1: http://webnodb.your_name.net4255.luxbulb.org/ => deployment webnodb
- url 2: http://webdb.your_name.net4255.luxbulb.org/ => deployment webdb
- Test your Ingress
- Update the schema of your infrastructure (ex. draw.io)
- Show the system
- Show the container IP address and the hostname of each container
- Show the container ports
Warning, please call the teacher before applying your Ingress.
- Create a HELM Chart to deploy the whole infrastructure
- Use ConfigMaps to store database hostname and port information
- Instead of a traditional deployment use a StafulSet to deploy your mongodb database
- Use a Persistant Volume to store the database content
- Stockage ressource : 0.1 Go
- Update the previous mongodb service with a headless service
- There should be only one replicat for the mongodb database
- Explain what is a StatefulSet and in which case it is usefull
- Explain what is a headless service, how pods are named with headless service
- Update your previous Helm chart accordinly
- Your first database in the statefulSet (example: mongo-0) should have a valid DNS hostname
- Kubernetes StatefulSet simply explained
- StatefulSets
- Tutorial/Howto about StatefulSet
- Kubernetes Volumes explained
- GKE Volumes persistants et provisionnement dynamique
- How to set pvc with statefulset in kubernetes?
- Update the version number in each of the HTML page of the respective website (webdb and webnodb) and rebuild their repective docker container (and bump up their version humber)
- Deploy your new container as a rolling update
- Create a deployment that spin a new pod when the CPU utilization of a pod cross a certain threasold (e.g.: 60% Utilization)
- Limit to maximum number of pods to be deploy to 10 pods
- Define Liveness Probe for each container in your Chart
- Becareful each application might need a specific type of probe
- Explain you choose a spefic type of probe for a given application
- Create a Network policy that restrict access to the database only to IP address corresponding to your Web Pods
- Test your network policy
- Create a master-slave architecture with mongodb
- Don't use already made Helm chart to achive this challenge
- Manualy configure each instance of the mongodb database to be part of a Replica Set (meaning that a given master database is replicated to all the slaves in the cluster)
- Replication Introduction
- Deploy a Replica Set
- Kubernetes StatefulSet explained
- Statefulsets | Deploying MongoDB clusters to Kubernetes
- Create a Helm Chart that deploy a master-slave architecture with mongodb
- Explain what is the advantage of updating your infrastructure with a redis cache ?
- Define your new infrastructure
- Use Hooks to configure your ConfigMaps before and after the deployment of your helm chart
- (Optional) Use hooks to save your database before updating helm chart