diff --git a/README.md b/README.md index a5233bb..66d94c5 100644 --- a/README.md +++ b/README.md @@ -248,16 +248,25 @@ Complete working examples are available in the `demo/` directory: - **[demo/spring](demo/spring)** — Spring Boot application - **[demo/helidon](demo/helidon)** — Helidon application -Run examples locally: +Run examples locally (from the repo root — the Gradle wrapper only exists there): ```bash # Plain Java -cd demo/java && ./gradlew run +./gradlew :demo:java:run # Spring Boot -cd demo/spring && ./gradlew bootRun --args='--spring.profiles.active=local' +./gradlew :demo:spring:bootRun --args='--spring.profiles.active=local' # Helidon -cd demo/helidon && ./gradlew run -Pmp.config.profile=local +./gradlew :demo:helidon:run -Pmp.config.profile=local +``` + +To exercise the AWS/Vault code paths without hitting real AWS or the +corporate Vault, use the local docker-compose stack (moto + Vault + +AppConfigData stub) documented in **[demo/local/README.md](demo/local/README.md)**: + +```bash +cd demo/local && docker compose up -d && source ./.env && cd ../.. +./gradlew :demo:spring:bootRun --args='--spring.profiles.active=moto' ``` ## Contributing diff --git a/demo/helidon/src/main/resources/META-INF/microprofile-config-moto.properties b/demo/helidon/src/main/resources/META-INF/microprofile-config-moto.properties new file mode 100644 index 0000000..a0d3546 --- /dev/null +++ b/demo/helidon/src/main/resources/META-INF/microprofile-config-moto.properties @@ -0,0 +1,25 @@ +# 'moto' profile overrides — point the demo at demo/local/docker-compose.yml +# (moto + vault) instead of real AWS / corporate Vault. +# +# Activate with: -Dmp.config.profile=moto (see run task in build.gradle) +# The profile is intentionally NOT called 'local' because Otto Config treats +# the 'local' profile as a signal to bypass the AWS/Vault sources entirely +# and fall back to file-based configuration (see +# CoreSourceFactory#isLocalProfile). +# +# Prerequisite: `cd demo/local && docker compose up -d && source ./.env` + +# SecretsManager: the SDK accepts a secret name for GetSecretValue, so we don't +# need the randomly-suffixed ARN moto generates. +otto.config.aws.secrets.arn=otto-config + +# Vault: use the local dev instance and approle auth (override the AWS/STS auth +# from the default microprofile-config.properties, which requires the corporate +# Vault). +otto.config.hashicorp.vault.url=http://localhost:8200 +otto.config.hashicorp.vault.auth.type=approle + +# Push-based refresh via the queue provisioned by init-moto.sh. moto uses the +# default AWS account id 123456789012. +otto.config.aws.change.notifications.enabled=true +otto.config.aws.change.notifications.queue.url=http://localhost:5000/123456789012/otto-config-config-changes diff --git a/demo/local/README.md b/demo/local/README.md new file mode 100644 index 0000000..6f38db5 --- /dev/null +++ b/demo/local/README.md @@ -0,0 +1,189 @@ +# Local demo infrastructure (moto + Vault) + +This directory brings up a fully offline replacement for the real AWS +account + corporate Vault that the demo services normally use. It runs: + +| Service | Image | Port | Purpose | +|---|---|---|---| +| `moto` | `motoserver/moto:latest` | `5000` | Mocks SecretsManager, SSM, SQS, EventBridge and STS. | +| `vault` | `hashicorp/vault:latest` | `8200` | Vault in dev mode (root token: `myroot`). | +| `appconfigdata-stub` | `eclipse-temurin:21-jdk` | `5001` | JDK-only Java HTTP server (single-file source, no build) that mimics the AWS AppConfigData data plane and serves the JSON files under `../terraform/`. Moto does not implement `appconfigdata`, so the SDK is pointed here for that one service via `AWS_ENDPOINT_URL_APPCONFIGDATA`. | +| `moto-init` | `amazon/aws-cli:latest` | – | Seeds moto with the Secrets Manager secret, SSM parameters and the SQS change-notification queue. | +| `vault-init` | `hashicorp/vault:latest` | – | Seeds Vault with the KV mount, policy, AppRole and demo secret; writes `role_id` / `secret_id` and AWS-endpoint env vars into `./.env`. | + +## Quick start + +```bash +# From the repo root +docker compose -f demo/local/docker-compose.yml up -d + +docker compose logs -f moto-init vault-init # Ctrl-C once both print "bootstrap done." + +# 2. Export the generated credentials + AWS endpoint override +source ./demo/local/.env + +# 3. Run one of the demos from the repo root (the Gradle wrapper only +# lives there) — it will talk to moto/vault, not real AWS. +./gradlew :demo:spring:bootRun --args='--spring.profiles.active=moto' +# or +./gradlew :demo:helidon:run -Pmp.config.profile=moto +# or +./gradlew :demo:java:run + +# 4. Tear it all down +cd demo/local && docker compose down -v +``` + +## How the demo picks up the mocks + +The AWS SDK v2 used by Otto Config natively honours the `AWS_ENDPOINT_URL` +environment variable, so pointing every AWS service at moto only needs the +following exports (all written to `./.env` by `init-vault.sh`): + +``` +AWS_ENDPOINT_URL=http://localhost:5000 # everything -> moto +AWS_ENDPOINT_URL_APPCONFIGDATA=http://localhost:5001 # AppConfigData -> stub +AWS_ACCESS_KEY_ID=test +AWS_SECRET_ACCESS_KEY=test +AWS_REGION=eu-central-1 +``` + +The service-specific `AWS_ENDPOINT_URL_APPCONFIGDATA` takes precedence +over the generic `AWS_ENDPOINT_URL` for the AppConfigData client only, +so AppConfig data-plane traffic goes to the stub while every other +service still hits moto. + +## The AppConfigData stub + +`appconfigdata-stub/AppConfigDataStub.java` is a self-contained, +JDK-only implementation of the two endpoints that Otto Config calls: + +* `POST /configurationsessions` — returns an opaque token that encodes + the requested profile (`properties` or `toggles`). +* `GET /configuration?configuration_token=…` — returns the raw JSON + content of the mounted file, or an empty body on subsequent polls + when the file hasn't changed (which Otto Config treats as "no + update"). + +It runs via JEP 330 single-file source launch (`java AppConfigDataStub.java`) +inside a stock `eclipse-temurin:21-jdk` container — no build step, no +third-party dependencies. Edit the JSON files under +`../terraform/appconfig_{properties,toggles}.json` and the next Otto +Config poll will pick up the change automatically. + +Vault-specific settings (URL, `approle` auth, generated `role_id` / +`secret_id`) come from: + +* `demo/spring/src/main/resources/application-moto.properties` +* `demo/helidon/src/main/resources/META-INF/microprofile-config-moto.properties` +* `$VAULT_ROLE_ID` / `$VAULT_SECRET_ID` exported by `./.env`. + +> **Why `moto` and not `local`?** Otto Config's `CoreSourceFactory` +> treats the profile names `local`, `test` and `integration-test` as a +> signal to bypass all AWS/Vault sources and read from the local +> `properties.json` file instead. Running against the mocked services +> requires a profile name outside that set. + +## Live-editing AppConfig via the stub + +The `../terraform/` directory is bind-mounted read-only into the stub +container at `/data`, so editing `appconfig_properties.json` / +`appconfig_toggles.json` on the host changes what the stub serves on the +*next* Otto Config poll. + +> **Why mount the parent directory and not the individual files?** On +> macOS Docker Desktop, single-file bind mounts pin the file's original +> inode: tools like `sed -i` (or any editor that atomically replaces the +> file) swap the inode on the host, and the container keeps serving the +> stale content. Mounting the parent directory sidesteps that. + +Try it: + +```bash +# 1. Start everything and run the Spring demo (repo root shell): +cd demo/local && docker compose up -d && source ./.env && cd ../.. +./gradlew :demo:spring:bootRun --args='--spring.profiles.active=moto' +``` + +Every 30 s the demo logs a line like: + +``` +DemoService : Property value for myKey1: myValue +DemoService : Toggle value for logging_enabled: false +``` + +In a second shell, edit the properties or toggles file and observe: + +```bash +# --- change a property --- +sed -i.bak 's/"myKey1": "myValue"/"myKey1": "hello-from-stub"/' \ + demo/terraform/appconfig_properties.json + +# --- flip the feature toggle (rewrite the file with jq or a small script; +# the multi-line JSON makes `sed` brittle here) --- +python3 -c "import json,pathlib; p=pathlib.Path('demo/terraform/appconfig_toggles.json'); d=json.loads(p.read_text()); d['values']['logging_enabled']['enabled']=True; p.write_text(json.dumps(d, indent=2))" + +# (Optional) trigger an immediate refresh instead of waiting up to 5 min +# for the background scheduler — this pushes an EventBridge-shaped event +# into the moto SQS queue that Otto Config polls every 10 s. +demo/local/notify-change.sh appconfig properties +demo/local/notify-change.sh appconfig toggles +``` + +Within ~10–40 s of running `notify-change.sh` the demo's next scheduled +`load()` will print the new values: + +``` +DemoService : Property value for myKey1: hello-from-stub +DemoService : Toggle value for logging_enabled: true +``` + +> **Only run one demo instance at a time.** All demo processes long-poll +> the same `otto-config-config-changes` SQS queue, and each SQS message +> is delivered to a single consumer. If you have leftover `bootRun` +> processes from earlier runs, they will steal change events from your +> current session — kill them first (`ps aux | grep spring:bootRun`). + +Under the hood: + +1. The bind mount means the stub sees the new file mtime immediately. +2. `notify-change.sh` sends an `aws.appconfig` EventBridge event to the + `otto-config-config-changes` SQS queue in moto. +3. Otto Config's `AwsChangeEventListener` (polling every 10 s) picks up + the message, matches it to the `AppConfigSource` for that profile, + and calls `refresh()` on it. +4. `AppConfigSource` re-hits the stub; the stub sees a newer mtime than + the last poll and returns the updated JSON content. +5. `@PropertyValue` / `@Value` bindings in `DemoService` observe the + new values on the next scheduled `load()`. + +The same trick works for `secrets` and `ssm`: + +```bash +# Change a secret, then notify: +aws --endpoint-url http://localhost:5000 secretsmanager put-secret-value \ + --secret-id otto-config \ + --secret-string '{"some_secret":"rotated!","some_other_secret":"still there"}' +demo/local/notify-change.sh secrets + +# Change an SSM parameter, then notify: +aws --endpoint-url http://localhost:5000 ssm put-parameter --overwrite \ + --name /search/develop/otto-config/config/some_ssm_value \ + --type String --value "changed-in-ssm" +demo/local/notify-change.sh ssm /search/develop/otto-config/config/some_ssm_value +``` + +## Re-seeding the mocks + +`init-moto.sh` deletes and re-creates every resource, so simply running + +```bash +docker compose up -d --force-recreate moto-init +``` + +resets the Secrets Manager secret, SSM parameters and SQS queue. + +The AppConfig properties/toggles are read live from +`../terraform/appconfig_properties.json` and +`../terraform/appconfig_toggles.json` by the stub — just edit the files +and the next Otto Config refresh cycle will pick up the changes. diff --git a/demo/local/appconfigdata-stub/AppConfigDataStub.java b/demo/local/appconfigdata-stub/AppConfigDataStub.java new file mode 100644 index 0000000..513d7ee --- /dev/null +++ b/demo/local/appconfigdata-stub/AppConfigDataStub.java @@ -0,0 +1,230 @@ +/* + * Tiny stand-in for the AWS AppConfigData data-plane API. Moto does not + * implement `appconfigdata` at all, so this stub is enough to let Otto + * Config's AppConfigSource pull hosted configuration content from local + * JSON files. + * + * Runs as a JEP 330 single-file source launch: + * + * java AppConfigDataStub.java + * + * No third-party dependencies — only the JDK's built-in HTTP server. + * + * Implements just enough of the API surface that AppConfigDataClient uses: + * + * POST /configurationsessions + * body: {"ApplicationIdentifier": "...", + * "EnvironmentIdentifier": "...", + * "ConfigurationProfileIdentifier": "properties" | "toggles"} + * response: {"InitialConfigurationToken": ""} + * + * GET /configuration?configuration_token= + * response body: raw JSON content + * response headers: + * Content-Type: application/json + * Next-Poll-Configuration-Token: + * Next-Poll-Interval-In-Seconds: 30 + * + * The profile identifier is encoded into the token so the GET can look up + * the right file. File mtime is tracked per token so a subsequent GET + * returns an empty body (which Otto Config treats as "no change") until + * the underlying JSON on disk changes. + */ +import com.sun.net.httpserver.HttpExchange; +import com.sun.net.httpserver.HttpHandler; +import com.sun.net.httpserver.HttpServer; + +import java.io.IOException; +import java.io.InputStream; +import java.net.InetSocketAddress; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Base64; +import java.util.LinkedHashMap; +import java.util.Map; +import java.util.UUID; +import java.util.concurrent.ConcurrentHashMap; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +public class AppConfigDataStub { + + private static final Map PROFILE_TO_FILE = Map.of( + "properties", "appconfig_properties.json", + "toggles", "appconfig_toggles.json" + ); + + private static final ConcurrentHashMap LAST_SEEN_MTIME = new ConcurrentHashMap<>(); + + public static void main(String[] args) throws IOException { + String host = envOrDefault("APPCONFIGDATA_STUB_HOST", "0.0.0.0"); + int port = Integer.parseInt(envOrDefault("APPCONFIGDATA_STUB_PORT", "5001")); + Path dataDir = Path.of(envOrDefault("APPCONFIGDATA_STUB_DATA_DIR", "/data")); + + HttpServer server = HttpServer.create(new InetSocketAddress(host, port), 0); + server.createContext("/configurationsessions", new StartSessionHandler()); + server.createContext("/configuration", new GetConfigurationHandler(dataDir)); + server.setExecutor(null); + server.start(); + System.out.printf("[appconfigdata-stub] listening on http://%s:%d, data dir = %s%n", host, port, dataDir); + } + + static class StartSessionHandler implements HttpHandler { + private static final Pattern PROFILE_FIELD = + Pattern.compile("\"ConfigurationProfileIdentifier\"\\s*:\\s*\"([^\"]+)\""); + + @Override public void handle(HttpExchange ex) throws IOException { + if (!"POST".equalsIgnoreCase(ex.getRequestMethod())) { respond(ex, 405, "{}"); return; } + String body; + try (InputStream in = ex.getRequestBody()) { + body = new String(in.readAllBytes(), StandardCharsets.UTF_8); + } + Matcher m = PROFILE_FIELD.matcher(body); + if (!m.find()) { respondJson(ex, 400, "{\"Message\":\"ConfigurationProfileIdentifier missing\"}"); return; } + String profile = m.group(1); + if (!PROFILE_TO_FILE.containsKey(profile)) { + respondJson(ex, 404, "{\"Message\":\"unknown profile '" + profile + "'\"}"); + return; + } + String token = encodeToken(profile); + respondJson(ex, 200, "{\"InitialConfigurationToken\":\"" + token + "\"}"); + } + } + + static class GetConfigurationHandler implements HttpHandler { + private final Path dataDir; + GetConfigurationHandler(Path dataDir) { this.dataDir = dataDir; } + + @Override public void handle(HttpExchange ex) throws IOException { + if (!"GET".equalsIgnoreCase(ex.getRequestMethod())) { respond(ex, 405, "{}"); return; } + Map q = parseQuery(ex.getRequestURI().getRawQuery()); + String token = q.get("configuration_token"); + if (token == null || token.isBlank()) { + respondJson(ex, 400, "{\"Message\":\"configuration_token required\"}"); + return; + } + String profile = decodeToken(token); + if (profile == null) { + respondJson(ex, 400, "{\"Message\":\"invalid token\"}"); + return; + } + String filename = PROFILE_TO_FILE.get(profile); + if (filename == null) { + respondJson(ex, 404, "{\"Message\":\"profile '" + profile + "' not found\"}"); + return; + } + Path file = dataDir.resolve(filename); + if (!Files.isRegularFile(file)) { + respondJson(ex, 404, "{\"Message\":\"file for profile '" + profile + "' not on disk\"}"); + return; + } + long mtime = Files.getLastModifiedTime(file).toMillis(); + Long previous = LAST_SEEN_MTIME.get(token); + String nextToken = encodeToken(profile); + LAST_SEEN_MTIME.put(nextToken, mtime); + ex.getResponseHeaders().add("Next-Poll-Configuration-Token", nextToken); + ex.getResponseHeaders().add("Next-Poll-Interval-In-Seconds", "30"); + if (previous != null && previous == mtime) { + // No change since last poll — Otto Config treats an empty body as "no update". + respondBytes(ex, 200, new byte[0], "application/json"); + return; + } + byte[] content = Files.readAllBytes(file); + // For an AWS.AppConfig.FeatureFlags profile the AppConfigData service + // returns only the resolved flag values (e.g. {"my_flag":{"enabled":true}}), + // not the full hosted document with 'flags' / 'values' / 'version' metadata. + // Otto Config's Toggles JsonCreator expects that resolved shape, so we + // strip the wrapper here for the toggles profile. + if ("toggles".equals(profile)) { + content = extractFeatureFlagValues(content); + } + respondBytes(ex, 200, content, "application/json"); + } + } + + /** + * Given the raw contents of an AppConfig FeatureFlags hosted configuration + * (shape: {"flags": {...}, "values": {...}, "version": "1"}), + * return just the bytes of the values object, matching what + * the real AWS AppConfigData service returns. Falls back to the input + * bytes if the wrapper isn't recognised. + */ + static byte[] extractFeatureFlagValues(byte[] hostedJson) { + String s = new String(hostedJson, StandardCharsets.UTF_8); + // Find "values" : { ... balanced ... } + Matcher key = Pattern.compile("\"values\"\\s*:\\s*\\{").matcher(s); + if (!key.find()) return hostedJson; + int start = key.end() - 1; // index of the opening '{' + int depth = 0; + boolean inString = false; + boolean escape = false; + for (int i = start; i < s.length(); i++) { + char c = s.charAt(i); + if (escape) { escape = false; continue; } + if (c == '\\' && inString) { escape = true; continue; } + if (c == '"') { inString = !inString; continue; } + if (inString) continue; + if (c == '{') depth++; + else if (c == '}') { + depth--; + if (depth == 0) { + return s.substring(start, i + 1).getBytes(StandardCharsets.UTF_8); + } + } + } + return hostedJson; + } + + // -------------------------------------------------------------------- helpers + + private static String encodeToken(String profile) { + String payload = "{\"p\":\"" + profile + "\",\"n\":\"" + UUID.randomUUID() + "\"}"; + return Base64.getUrlEncoder().withoutPadding() + .encodeToString(payload.getBytes(StandardCharsets.UTF_8)); + } + + private static String decodeToken(String token) { + try { + byte[] raw = Base64.getUrlDecoder().decode(token); + String s = new String(raw, StandardCharsets.UTF_8); + Matcher m = Pattern.compile("\"p\"\\s*:\\s*\"([^\"]+)\"").matcher(s); + return m.find() ? m.group(1) : null; + } catch (RuntimeException e) { + return null; + } + } + + private static Map parseQuery(String raw) { + Map out = new LinkedHashMap<>(); + if (raw == null || raw.isEmpty()) return out; + for (String pair : raw.split("&")) { + int eq = pair.indexOf('='); + if (eq < 0) out.put(pair, ""); + else out.put(pair.substring(0, eq), java.net.URLDecoder.decode(pair.substring(eq + 1), StandardCharsets.UTF_8)); + } + return out; + } + + private static void respond(HttpExchange ex, int status, String body) throws IOException { + respondBytes(ex, status, body.getBytes(StandardCharsets.UTF_8), null); + } + private static void respondJson(HttpExchange ex, int status, String body) throws IOException { + respondBytes(ex, status, body.getBytes(StandardCharsets.UTF_8), "application/json"); + } + private static void respondBytes(HttpExchange ex, int status, byte[] body, String contentType) throws IOException { + if (contentType != null) ex.getResponseHeaders().add("Content-Type", contentType); + ex.sendResponseHeaders(status, body.length); + if (body.length > 0) { + ex.getResponseBody().write(body); + } + ex.getResponseBody().close(); + System.out.printf("[appconfigdata-stub] %s %s -> %d (%d bytes)%n", + ex.getRequestMethod(), ex.getRequestURI(), status, body.length); + } + + private static String envOrDefault(String key, String fallback) { + String v = System.getenv(key); + return (v == null || v.isEmpty()) ? fallback : v; + } +} diff --git a/demo/local/docker-compose.yml b/demo/local/docker-compose.yml new file mode 100644 index 0000000..64ce833 --- /dev/null +++ b/demo/local/docker-compose.yml @@ -0,0 +1,125 @@ +# Local infrastructure for the demo services. +# +# Starts: +# - moto : an AWS mock exposing SecretsManager, SSM, SQS, EventBridge +# and STS on http://localhost:5000 (moto has no AppConfigData +# implementation — see appconfigdata-stub below). +# - vault : Hashicorp Vault dev-mode instance on http://localhost:8200 +# - appconfigdata-stub : tiny JDK-only Java HTTP server that mimics the +# AWS AppConfigData data plane and serves the JSON files in +# ../terraform/. Runs on http://localhost:5001 and is +# addressed via AWS_ENDPOINT_URL_APPCONFIGDATA so the SDK +# routes only AppConfigData calls to it and everything else +# still goes to moto. +# - moto-init : seeds moto with the Secrets Manager secret, the SSM +# parameters and the SQS change-notification queue. +# - vault-init : seeds Vault with the KV secret, AppRole policy/role and +# writes role_id / secret_id into ./.env so the demo apps +# can pick them up +# +# Usage: +# cd demo/local +# docker compose up -d +# # wait until the *-init containers finished (docker compose ps) +# source ./.env +# cd ../.. # back to repo root, where the gradle wrapper lives +# ./gradlew :demo:spring:bootRun --args='--spring.profiles.active=moto' +# ./gradlew :demo:helidon:run -Pmp.config.profile=moto +# ./gradlew :demo:java:run +# +# Tear down: +# docker compose down -v + +services: + moto: + image: motoserver/moto:latest + container_name: otto-config-moto + ports: + - "5000:5000" + environment: + MOTO_SERVICE: "" # serve all services + MOTO_LOG_LEVEL: WARNING + healthcheck: + # Poll frequently while starting up, then once a minute in steady state + # so healthcheck traffic doesn't drown out real requests in the logs. + test: ["CMD", "python", "-c", "import urllib.request,sys;sys.exit(0) if urllib.request.urlopen('http://localhost:5000/moto-api/').status==200 else sys.exit(1)"] + interval: 60s + timeout: 3s + retries: 3 + start_period: 30s + start_interval: 1s + + vault: + image: hashicorp/vault:latest + container_name: otto-config-vault + cap_add: + - IPC_LOCK + environment: + VAULT_DEV_ROOT_TOKEN_ID: myroot + VAULT_DEV_LISTEN_ADDRESS: 0.0.0.0:8200 + VAULT_ADDR: http://127.0.0.1:8200 + ports: + - "8200:8200" + healthcheck: + test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8200/v1/sys/health"] + interval: 60s + timeout: 3s + retries: 3 + start_period: 30s + start_interval: 1s + + appconfigdata-stub: + image: eclipse-temurin:21-jdk + container_name: otto-config-appconfigdata-stub + working_dir: /app + command: ["java", "AppConfigDataStub.java"] + environment: + APPCONFIGDATA_STUB_DATA_DIR: /data + APPCONFIGDATA_STUB_PORT: "5001" + ports: + - "5001:5001" + volumes: + - ./appconfigdata-stub/AppConfigDataStub.java:/app/AppConfigDataStub.java:ro + # Mount the parent directory (not the individual files) so host-side + # edits that replace the inode — like `sed -i` on macOS — are picked + # up. Individual file bind mounts on Docker Desktop for Mac cache + # the original inode and keep serving the old content. + - ../terraform:/data:ro + healthcheck: + test: ["CMD-SHELL", "exec 3<>/dev/tcp/127.0.0.1/5001 && printf 'GET /configuration HTTP/1.0\\r\\n\\r\\n' >&3 && head -n1 <&3"] + interval: 60s + timeout: 3s + retries: 3 + start_period: 30s + start_interval: 1s + + moto-init: + image: amazon/aws-cli:latest + container_name: otto-config-moto-init + depends_on: + moto: + condition: service_healthy + entrypoint: ["/bin/bash", "/init/init-moto.sh"] + environment: + AWS_ACCESS_KEY_ID: test + AWS_SECRET_ACCESS_KEY: test + AWS_DEFAULT_REGION: eu-central-1 + AWS_ENDPOINT_URL: http://moto:5000 + volumes: + - ./init-moto.sh:/init/init-moto.sh:ro + restart: "no" + + vault-init: + image: hashicorp/vault:latest + container_name: otto-config-vault-init + depends_on: + vault: + condition: service_healthy + entrypoint: ["/bin/sh", "/init/init-vault.sh"] + environment: + VAULT_ADDR: http://vault:8200 + VAULT_TOKEN: myroot + volumes: + - ./init-vault.sh:/init/init-vault.sh:ro + - ./:/out + restart: "no" diff --git a/demo/local/init-moto.sh b/demo/local/init-moto.sh new file mode 100755 index 0000000..2406e7a --- /dev/null +++ b/demo/local/init-moto.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +# Seeds the moto server with the AWS resources the demo services expect: +# - Secrets Manager secret 'otto-config' +# - SSM parameters under /search/develop/otto-config +# - SQS queue 'otto-config-config-changes' +# +# AppConfig is *not* seeded here — moto does not implement the +# AppConfigData data plane. The appconfigdata-stub container serves the +# hosted configuration content instead. +# +# Idempotent: re-running is safe (deletes and re-creates the secret / +# parameters / queue). +set -euo pipefail + +SERVICE="otto-config" +SSM_PREFIX="/search/develop/otto-config" +QUEUE_NAME="otto-config-config-changes" + +aws() { + command aws --endpoint-url "${AWS_ENDPOINT_URL}" "$@" +} + +echo ">>> Waiting for moto to accept API calls..." +for i in $(seq 1 30); do + if aws sts get-caller-identity >/dev/null 2>&1; then break; fi + sleep 1 +done + +######################################## +# Secrets Manager +######################################## +echo ">>> Seeding Secrets Manager secret '${SERVICE}'" +aws secretsmanager delete-secret --secret-id "${SERVICE}" --force-delete-without-recovery >/dev/null 2>&1 || true +SECRET_ARN=$(aws secretsmanager create-secret \ + --name "${SERVICE}" \ + --secret-string '{"some_secret":"some very secret value","some_other_secret":"some other secret value"}' \ + --query 'ARN' --output text) +echo " secret arn = ${SECRET_ARN}" + +######################################## +# SSM Parameter Store +######################################## +echo ">>> Seeding SSM parameters under ${SSM_PREFIX}" +aws ssm put-parameter --name "${SSM_PREFIX}/config/some_ssm_value" --type String --value "hello-from-ssm" --overwrite >/dev/null +aws ssm put-parameter --name "${SSM_PREFIX}/config/another_ssm_value" --type String --value "another-hello" --overwrite >/dev/null + +######################################## +# SQS change-notification queue +######################################## +echo ">>> Seeding SQS queue '${QUEUE_NAME}'" +QUEUE_URL=$(aws sqs create-queue --queue-name "${QUEUE_NAME}" --query 'QueueUrl' --output text) +echo " queue url = ${QUEUE_URL}" + +echo ">>> moto bootstrap done." diff --git a/demo/local/init-vault.sh b/demo/local/init-vault.sh new file mode 100755 index 0000000..bb02fbf --- /dev/null +++ b/demo/local/init-vault.sh @@ -0,0 +1,65 @@ +#!/bin/sh +# Seeds Vault dev-mode with the KV mount, policy and AppRole the demo apps +# expect. Writes role_id / secret_id to /out/.env so the demo apps running +# on the host can pick them up. +set -eu + +VAULT_ADDR="${VAULT_ADDR:-http://vault:8200}" +export VAULT_ADDR +export VAULT_TOKEN="${VAULT_TOKEN:-myroot}" + +echo ">>> Waiting for Vault to be ready..." +for i in $(seq 1 30); do + if vault status >/dev/null 2>&1; then break; fi + sleep 1 +done + +echo ">>> Enabling KV v2 secrets engine at cftsearch/" +vault secrets disable cftsearch >/dev/null 2>&1 || true +vault secrets enable -path=cftsearch kv-v2 + +echo ">>> Enabling AppRole auth method" +vault auth disable approle >/dev/null 2>&1 || true +vault auth enable approle + +echo ">>> Writing policy otto-config-policy" +cat <<'EOF' | vault policy write otto-config-policy - +path "cftsearch/data/*" { + capabilities = ["create", "read", "update", "delete", "list"] +} +path "cftsearch/metadata/*" { + capabilities = ["list", "read"] +} +EOF + +echo ">>> Creating AppRole otto-config" +vault write auth/approle/role/otto-config \ + policies="otto-config-policy" \ + token_ttl=6m \ + token_max_ttl=10m >/dev/null + +ROLE_ID=$(vault read -field=role_id auth/approle/role/otto-config/role-id) +SECRET_ID=$(vault write -field=secret_id -f auth/approle/role/otto-config/secret-id) + +echo ">>> Seeding secret cftsearch/service/otto-config/develop/auth" +vault kv put cftsearch/service/otto-config/develop/auth \ + auth_client_id=admin \ + auth_client_secret=secret >/dev/null + +echo ">>> Writing /out/.env" +cat > /out/.env <>> vault bootstrap done. role_id=${ROLE_ID}" diff --git a/demo/local/notify-change.sh b/demo/local/notify-change.sh new file mode 100755 index 0000000..8d53ea1 --- /dev/null +++ b/demo/local/notify-change.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# Push an EventBridge-shaped change event into the moto-hosted SQS queue +# so Otto Config's SQS listener picks it up and forces an immediate +# refresh of the given source. +# +# Without this you have to wait for the safety-net scheduler to fire +# (every 5 minutes for AppConfig / Secrets / SSM in the Spring demo). +# +# Usage: +# ./notify-change.sh appconfig [properties|toggles] +# ./notify-change.sh secrets +# ./notify-change.sh ssm /search/develop/otto-config/config/some_ssm_value +# +# Requires: aws CLI on your PATH and the local docker-compose stack up. +set -euo pipefail + +# Load .env so AWS_ENDPOINT_URL / creds are set even in a fresh shell. +if [ -f "$(dirname "$0")/.env" ]; then + # shellcheck disable=SC1091 + . "$(dirname "$0")/.env" +fi + +: "${AWS_ENDPOINT_URL:=http://localhost:5000}" +: "${AWS_REGION:=eu-central-1}" +QUEUE_URL="${AWS_ENDPOINT_URL}/123456789012/otto-config-config-changes" + +kind="${1:-}" +arg="${2:-}" + +case "$kind" in + appconfig) + profile="${arg:-properties}" + body=$(cat <&2 + exit 2 + ;; +esac + +aws --endpoint-url "${AWS_ENDPOINT_URL}" sqs send-message \ + --queue-url "${QUEUE_URL}" \ + --message-body "${body}" \ + --output text >/dev/null + +echo "sent ${kind} change event to ${QUEUE_URL}" diff --git a/demo/spring/src/main/resources/application-moto.properties b/demo/spring/src/main/resources/application-moto.properties new file mode 100644 index 0000000..d96442f --- /dev/null +++ b/demo/spring/src/main/resources/application-moto.properties @@ -0,0 +1,24 @@ +# 'moto' profile overrides — point the demo at demo/local/docker-compose.yml +# (moto + vault) instead of real AWS / corporate Vault. +# +# Activate with: --spring.profiles.active=moto +# The profile is intentionally NOT called 'local' because Otto Config treats +# the 'local' profile as a signal to bypass the AWS/Vault sources entirely +# and fall back to file-based configuration (see +# CoreSourceFactory#isLocalProfile). +# +# Prerequisite: `cd demo/local && docker compose up -d && source ./.env` + +# SecretsManager: the SDK accepts a secret name for GetSecretValue, so we don't +# need the randomly-suffixed ARN moto generates. +otto.config.aws.secrets.arn=otto-config + +# Vault: use the local dev instance and approle auth (override the AWS/STS auth +# from application.properties, which requires the corporate Vault). +otto.config.hashicorp.vault.url=http://localhost:8200 +otto.config.hashicorp.vault.auth.type=approle + +# Push-based refresh via the queue provisioned by init-moto.sh. moto uses the +# default AWS account id 123456789012. +otto.config.aws.change.notifications.enabled=true +otto.config.aws.change.notifications.queue.url=http://localhost:5000/123456789012/otto-config-config-changes