docs: document how to use ES/OS exporter with self signed certificates

[npepinpe]

Description

Document how to configure the ES/OS exporters with self signed certificates.

When should this change go live?

• This is a bug fix, security concern, or something that needs urgent release support. (add bug or support label)
• This is already available but undocumented and should be released within a week. (add available & undocumented label)
• This is on a specific schedule and the assignee will coordinate a release with the DevEx team. (create draft PR and/or add hold label)
• This is part of a scheduled alpha or minor. (add alpha or minor label)
• There is no urgency with this change (add low prio label)

PR Checklist

• My changes are for an upcoming minor release and:
  • are in the /docs directory (version 8.8).
  • are in the /versioned_docs/version-8.7/ directory (version 8.7).
• My changes are for an already released minor and are in a /versioned_docs directory.

• I added a DRI, team, or delegate as a reviewer for technical accuracy and grammar/style:
  • Engineering team review
  • Technical writer review via @camunda/tech-writers unless working with an embedded writer.

[github-actions]

👋 🤖 ✅ Looks like the changes were ported across versions, nice job! 🎉

You can read more about the versioning within our docs in our documentation guidelines.

[npepinpe]

@ChrisKujawa - I did not finish yet, still need to add to OS and versioned docs, but as it will be mostly copy paste, I would use an early review on the "main" content.

I tested it by first creating a self signed certificate, without any password for the key:

openssl genrsa -out /tmp/key.pem 2048
openssl req -new -sha256 -key /tmp/key.pem -out /tmp/csr.csr
openssl req -x509 -sha256 -days 365 -key /tmp/key.pem -in /tmp/csr.csr -out /tmp/certificate.pem

Make sure to set localhost as the common name (aka server FQDN), otherwise hostname verification will not work, and you'll have to turn it off.

Then creating the trust store as described in the docs. I used changeme as password.

keytool -importkeystore -srckeystore $JAVA_HOME/lib/security/cacerts -destkeystore /tmp/zeebeTrustStore.jks -srcstoretype PKCS12 -deststoretype JKS
keytool -import -alias localhost -keystore /tmp/zeebeTrustStore.jks -file /tmp/certificate.pem

Then running the following test. You can save the file in the elasticsearch-exporter module, e.g. under zeebe/exporters/elasticsearch-exporter/src/test/java/io/camunda/zeebe/exporter/SelfSignedIT.java

/*
 * Copyright Camunda Services GmbH and/or licensed to Camunda Services GmbH under
 * one or more contributor license agreements. See the NOTICE file distributed
 * with this work for additional information regarding copyright ownership.
 * Licensed under the Zeebe Community License 1.1. You may not use this file
 * except in compliance with the Zeebe Community License 1.1.
 */
package io.camunda.zeebe.exporter;

import static org.assertj.core.api.Assertions.assertThatNoException;

import io.camunda.zeebe.test.util.testcontainers.TestSearchContainers;
import io.micrometer.core.instrument.simple.SimpleMeterRegistry;
import io.netty.handler.ssl.util.SelfSignedCertificate;
import java.io.IOException;
import java.nio.file.Path;
import java.security.cert.CertificateException;
import java.util.Objects;
import java.util.UUID;
import org.agrona.LangUtil;
import org.junit.jupiter.api.Test;
import org.testcontainers.elasticsearch.ElasticsearchContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.MountableFile;

@Testcontainers
final class SelfSignedIT {

  @Container
  private static final ElasticsearchContainer CONTAINER =
      TestSearchContainers.createDefeaultElasticsearchContainer()
          .withEnv("xpack.license.self_generated.type", "trial")
          .withEnv("xpack.security.enabled", "true")
          .withEnv("xpack.security.http.ssl.enabled", "true")
          .withEnv("xpack.security.http.ssl.certificate", "cert.pem")
          .withEnv("xpack.security.http.ssl.key", "key.pem")
          .withEnv("xpack.security.authc.anonymous.username", "anon")
          .withEnv("xpack.security.authc.anonymous.roles", "superuser")
          .withEnv("xpack.security.authc.anonymous.authz_exception", "true")
          .withCopyFileToContainer(
              MountableFile.forHostPath("/tmp/certificate.pem", 0777),
              "/usr/share/elasticsearch/config/cert.pem")
          .withCopyFileToContainer(
              MountableFile.forHostPath("/tmp/key.pem", 0777),
              "/usr/share/elasticsearch/config/key.pem");

  @Test
  void shouldConnectWithSelfSignedCertificate() throws IOException {
    // when
    // force recreating the client
    final var config = new ElasticsearchExporterConfiguration();
    config.index.prefix = UUID.randomUUID() + "-test-record";
    config.url = "https://" + CONTAINER.getHttpHostAddress();

    // when
    try (final var client = new ElasticsearchClient(config, new SimpleMeterRegistry())) {
      // then
      assertThatNoException().isThrownBy(client::putComponentTemplate);
    }
  }
}

If you run it as is, it will fail due to the certificate. Then edit the IntelliJ configuration, update the VM options, and add -Djavax.net.ssl.trustStore=/tmp/zeebeTrustStore.jks -Djavax.net.ssl.trustStorePassword=changeme, and run again.

[npepinpe]

FYI for tech writers: this should be released as soon as possible, when approved by both engineering and you :)

[conceptualshark]

I've updated the formatting somewhat here, and this looks good to me! Happy to merge once the technical review is complete.

[conceptualshark]

@npepinpe I see the request to merge quickly, and the versions selected - should I also backport this to all prior versions (8.3 - 8.7) once all reviews are finished?

[npepinpe]

No, I can do that, it'll be mostly copy paste.

[remcowesterhoud]

I haven't verified if it actually works, but it seems like it should.

[npepinpe]

@conceptualshark - I've backported the docs and also copied them for the OS exporter (which has the same problem).

If it all looks good from your side, we can merge 👍 As this is for a customer, it would be great if we could do this ASAP. Please let me know whenever this is done and available for customers/users.

[conceptualshark]

All good! I'll merge and can start a release going shortly, and will comment here when it's live.

[conceptualshark]

@npepinpe This should all be live!