Skip to content

Latest commit

 

History

History
117 lines (96 loc) · 5.43 KB

README.md

File metadata and controls

117 lines (96 loc) · 5.43 KB
Raw

Okteto Elasticsearch

Deploy Elasticsearch 8.x cluster on Okteto Cloud

✨ Features

🚀 Steps

  • Create an Okteto account, install and configure the Okteto CLI
  • Clone the okteto-elasticsearch repo
  • Generate the ES certificates:
    • Start Docker and run $ bash scripts/certgen-launcher.sh
  • Deploy on Okteto
    • Run $ okteto deploy --build
    • Check the created endpoint from the previous output
  • Call the ES endpoint:
    • Note: if not configured [1], <your-password> value is changeme
    $ curl -XGET -u elastic:<your-password> https://<your-endpoint-created>.cloud.okteto.net/_cat/nodes\?v

# Example:
$ curl -XGET -u elastic:changeme https://es01-http-mynamespace.cloud.okteto.net/_cat/nodes\?v
ip          heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
10.8.38.167            7          62  32    1.69    1.41     0.93 cdfhilmrstw *      es02
10.8.38.166           10          60  27    1.69    1.41     0.93 cdfhilmrstw -      es01
10.8.38.168           11          62  36    1.69    1.41     0.93 cdfhilmrstw -      es03
  • Enjoy your cluster!
    • Do you want to use Kibana? see [2]
    • Don't waste free resources, if you don't need the cluster tear down everything with $ okteto destroy -v

✍️ Notes

  • Security is provided by:
  • Kubernetes
    • Instead of declaring directly the GKE ingress, we will use the Okteto provided auto SSL
      • Through the dev.okteto.com/auto-ingress: "true" annotation
    • We will create one ClusterIP for each note for port 9300
      • Because ES uses that as the default port for internode communication

🔧 How to

  • [1] Change the default Elasticsearch password:
    • Generate the base64 new password
      • $ echo "NEW_PASSWORD" | tr -d \\n | base64 -w 0
    • Open the the k8s/elasticsearch.yml file
      • Use the generated value to replace the ELASTIC_PASSWORD value of the Secret component
  • [2] Run Kibana locally
    • 🚧 Currently WIP, waiting this ES issue will be resolved
    • Run kibana locally and connect with Okteto cluster:
      • We run the docker locally to don't waste the okteto cloud resources

⚒️ Okteto

Development environment

  • We could test the internode network thanks to Okteto development environment 
    # Start the busybox-curl pod
$ okteto up

# The pod is mounted with all the local files, including the certificates:
> ls -l /okteto/
Dockerfile  README.md   certs       k8s         okteto.yml  scripts

# The pod is deployed into the cluster and could use the certificates:
> curl -u elastic:changeme es-http:9200
{
  "name" : "es01",
  "cluster_name" : "okteto-cluster",
...

> nc -vz es01 9300
es01 (10.153.19.186:9300) open

Sleeping system

  • Q: "How can I restart a sleeping development environment?" - link
    • A: Visit any of the public endpoints of your development environment

Okteto useful commands

# Log into the cluster
$ okteto init

# Deploy the local `okteto.yml` - wait 5/10m
$ okteto deploy --wait

# Activate a development container
# > https://www.okteto.com/docs/reference/cli/#up
$ okteto up

# Create kubectl context to Okteto cloud
$ okteto kubeconfig
$ kubectl get po

🛂 Disclaimer

This repository is built for side-project purposes and no warranties are provided. Activities to keep in mind before using in production environments includes but are not limited to:

  • We will arbitrarily expose the es01 node as API server:
    • So we don't have load balancing between the API requests
    • There is no guarantee that es01 isn't chosen as the master node
  • Create a more robust ES architecture with dedicated ES master nodes
  • Fine-tune the ES nodes' roles and HW requirements
  • All the points listed in the "TODO" section

💤 TODOs

  • mount data volumes on /usr/share/elasticsearch/data to ensure the pods' data persistence
  • avoid storing the ES password in the yaml file
  • integrate the elasticsearch-readonlyrest-plugin