Make Elasticsearch CA configurable

[bartjkdp]

This change makes the Elasticsearch CA configurable and adds the possibility to configure Xpack security. This improves the security of Signalen as it allows encryption between the backend and Elasticsearch and authentication on the Elasticsearch server.

To try this locally, use:

mkdir certs
docker run --name elastic-helm-charts-certs -i -w /app docker.elastic.co/elasticsearch/elasticsearch:7.17.3 /bin/sh -c "elasticsearch-certutil ca --out /app/elastic-stack-ca.p12 --pass '' && elasticsearch-certutil cert --name elasticsearch --dns elasticsearch --ca /app/elastic-stack-ca.p12 --pass '' --ca-pass '' --out /app/elastic-certificates.p12"
docker cp elastic-helm-charts-certs:/app/elastic-certificates.p12 ./certs
docker rm -f elastic-helm-charts-certs
openssl pkcs12 -nodes -passin pass:'' -in certs/elastic-certificates.p12 -out certs/elastic-certificate.pem
openssl x509 -outform der -in certs/elastic-certificate.pem -out certs/elastic-certificate.crt

Adjust the docker-compose.yml file:

services:
  api:
    build:
      context: .
    ports:
      - "8000:8000"
    depends_on:
      dex:
        condition: service_started
      mailhog:
        condition: service_started
      database:
        condition: service_healthy
      elasticsearch:
        condition: service_started
      celery:
        condition: service_started
      celery_beat:
        condition: service_started
    env_file:
      - docker-compose/environments/.api
    volumes:
      - ./app:/app
      - ./dwh_media:/dwh_media
      - ./docker-compose/scripts/initialize.sh:/initialize.sh
      - ./certs:/certs
    command:
      - /initialize.sh

...

  elasticsearch:
    image: elasticsearch:7.17.9
    shm_size: '512m'
    command: elasticsearch
    environment:
      - http.host=0.0.0.0
      - transport.host=127.0.0.1
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12
      - xpack.security.http.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - es-data:/usr/share/elasticsearch/data
      - ./certs:/usr/share/elasticsearch/config/certs

And adjust the docker-compose/environments/.api file:

ELASTICSEARCH_HOST=https://signalen:something-secret@elasticsearch:9200/
ELASTICSEARCH_CA_BUNDLE=/certs/elastic-certificate.pem

Start the environment:

docker-compose up -d

And create the Elasticsearch user:

docker-compose exec elasticsearch sh
bin/elasticsearch-users useradd signalen
<enter password>
bin/elasticsearch-users roles -a superuser signalen

[4c0n]

As communicated through Slack, this improvement currently does not have our priority. As such we don't have time available to test this and will perhaps take a look at this at a future time.
Should this be an issue for any of the municipalities that are currently using the application, then priority can be requested through the VNG product owner route.

[bartjkdp]

Thanks for the response @4c0n. I am currently requesting priority as I think this is an important security improvement for Signalen.