Grafana DevLake Metrics

A project to collect, analyze, and visualize DevOps metrics using Grafana and DevLake.

Table of Contents

1. Overview
2. Features
3. Prerequisites
4. Getting Started
   • Local Deployment
   • Azure Deployment
5. Use of GitHub Actions
   • Deploy Pods to Azure
   • Destroy Pods on Azure
6. Project Structure
7. Contributing
8. License
9. References

Overview

This project integrates DevLake with Grafana to provide insightful dashboards for DevOps metrics, enabling teams to monitor and improve their software development processes. Additionally, the project includes monitoring of the Kubernetes cluster itself, providing insights into cluster health, resource usage, and performance.

Features

• Collects data from various DevOps tools (e.g., GitHub, Jira)
• Processes and transforms data using DevLake
• Visualizes metrics and trends in Grafana dashboards
• Customizable and extensible for different workflows
• Deployable to Azure Kubernetes Service (AKS)

Prerequisites

• Azure account with permissions to create resources (Secrets in GitHub)
• Install kubernetes provider ex. Docker Desktop
• Installed terraform
• Access to supported data sources (e.g., GitHub, Jira)

Getting Started

Local Deployment

1. Clone the repository:

   • Clone the repository to your local machine and navigate to the project directory:

   git clone https://github.com/whiteduck-classroom/Grafana-DevLake-Metrics.git
   cd Grafana-DevLake-Metrics

2. Navigate to the services directory:

   • Since deployments are managed with GitHub Actions, navigate to the services directory where the Terraform configuration is located:

   cd services

3. Initialize and apply Terraform:

   • Initialize Terraform to download the required providers and apply the configuration to set up the local environment:

   terraform init
   terraform apply

4. Access Grafana:

   • Once the deployment is complete, you can access Grafana in your browser. If you are running the deployment locally, you might need to port-forward the Grafana service to access it:

     kubectl port-forward svc/grafana 3000:3000 -n grafana

   • After port-forwarding, open your browser and navigate to:

     • URL: http://localhost:3000

   • Default credentials:

     • Username: admin
     • Password: admin

---

Azure Deployment

This project can be deployed to Azure Kubernetes Service (AKS) for production use.

But be aware that this Project is designed to be deployed over GitHub Actions

1. Set up Azure resources:

   • Create a resource group:
     • replace name-resource-group with the name of your

     az group create --name <name-resource-group> --location westeurope

   • Create an AKS cluster:

     az aks create --resource-group <name-resource-group> --name devlake-aks --node-count 1 --enable-managed-identity --generate-ssh-keys

2. Configure Kubernetes access:

   • Get AKS credentials:

     az aks get-credentials --resource-group <name-resource-group> --name devlake-aks

   • Verify access:

     kubectl get nodes

3. Deploy the stack to AKS:

   • Use Terraform to deploy the services:

     terraform init
     terraform apply

4. Access Grafana and DevLake-ui:

   • Grafana: http://:3000
   • DevLake: http://:4000

---

Use of GitHub Actions

This project includes GitHub Actions for deploying and managing resources on Microsoft Azure.

1. Deploy Pods to Azure

1. Navigate to the Actions tab in your GitHub repository.
2. Select the Terraform Deploy workflow.
3. Click on Run workflow.
4. Configure the deployment options:
   • Number of Nodes: Specify how many nodes you want to create (default: 1).
   • Azure Resource Group: Enter the name of your Azure Resource Group.
5. Press Run workflow to start the deployment process.

---

2. Destroy Pods on Azure

1. Navigate to the Actions tab in your GitHub repository.
2. Select the Terraform Destroy workflow.
3. Click on Run workflow.
4. Confirm the destroy action:
   • Enter destroy in the confirmation field.
5. Choose what to destroy:
   • Destroy all pods:
     • Leave the dropdown as default and press Run workflow.
   • Destroy specific pods:
     • Select the specific option from the dropdown.
     • Specify the name of the pod(s) you want to delete.
     • Press Run workflow.

Project Structure

• /azure-aks/ - Azure AKS Terraform config
• /services/ - DevLake and Kube-Prometheus config

Contributing

Contributions are welcome! Please open issues or pull requests for improvements.

License

References

• Apache DevLake Kubernetes
• Grafana-Prometheus Kubernetes