Devcontainer for local dev #93

jakewatson-bristol · 2025-07-28T15:46:45Z

add a dev container with kind for a local k8s cluster

also adds just for running useful commands, once in the dev container run just and it will list the avalible args

just rebuild-cluster will rebuild the local kind cluster (including generate a new kubeconfig)
`just pulumi-init' will help people get pulumi set up (set up local login, create a stack, help user update the config for their needs)

let me know your thoughts

craddm · 2025-07-30T13:50:15Z

It failed on the postCreateCommand step for me:

[8948 ms] Start: Run in container: /bin/sh -c bash .devcontainer/post-create.sh
================================================================================
>>> Setting up K3s development configuration
================================================================================
cp: cannot stat '/kubeconfig/kubeconfig.yaml': No such file or directory

What's next:
    Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug 0d0858efc19b0e0347effd7aaafb7269744c86c59f49a8ddf87a2736ebc0710f
    Learn more at https://docs.docker.com/go/debug-cli/
[9242 ms] Stop (294 ms): Run in container: /bin/sh -c bash .devcontainer/post-create.sh
[9242 ms] postCreateCommand from devcontainer.json failed with exit code 1. Skipping any further user-provided commands.
Done. Press any key to close the terminal.

awalford16 · 2025-08-08T08:51:56Z

Worth adding helm to the container?

craddm · 2025-08-08T09:12:14Z

@awalford16 I'd agree with that; but I also know @jakewatson-bristol has done some more work on this that isn't reflected in the PR at the moment because of some merging problems

…ame before class names

craddm · 2025-08-12T08:58:58Z

kube.yaml

We should delete this file

craddm · 2025-08-12T09:04:10Z

.devcontainer/devcontainer.Dockerfile

+# install KInD
+RUN if [ "$(uname -m)" = "x86_64" ]; then \
+        curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.29.0/kind-linux-amd64; \
+    elif [ "$(uname -m)" = "aarch64" ]; then \
+        curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.29.0/kind-linux-arm64; \
+    fi && \
+    chmod +x ./kind && \
+    mv ./kind /usr/local/bin/kind


Since we're not using Kind any more, we can delete this

craddm · 2025-08-12T09:04:32Z

.devcontainer/devcontainer.Dockerfile

+    chmod +x k3d-linux-* && \
+    mv k3d-linux-* /usr/local/bin/k3d
+
+# Istall k9s


Suggested change

# Istall k9s

# Install k9s

craddm · 2025-08-12T09:05:02Z

.devcontainer/devcontainer.Dockerfile

+    tar xzvf hubble-linux-${HUBBLE_ARCH}.tar.gz -C /usr/local/bin && \
+    rm hubble-linux-*.tar.gz hubble-linux-*.tar.gz.sha256sum
+
+# Install ArgoCD CLI


Suggested change

# Install ArgoCD CLI

# Install Argo Workflows CLI

craddm · 2025-08-12T09:05:56Z

.devcontainer/devcontainer.Dockerfile

+
+# Install ArgoCD CLI
+ARG ARGO_OS="linux"
+RUN curl -sLO "https://github.com/argoproj/argo-workflows/releases/download/v3.7.0/argo-$ARGO_OS-amd64.gz"


We define the version as a constant at the start of the script, but don't use it here

Suggested change

RUN curl -sLO "https://github.com/argoproj/argo-workflows/releases/download/v3.7.0/argo-$ARGO_OS-amd64.gz"

RUN curl -sLO "https://github.com/argoproj/argo-workflows/releases/download/$ARGO_VERSION/argo-$ARGO_OS-amd64.gz"

craddm · 2025-08-12T09:06:05Z

.devcontainer/devcontainer.Dockerfile

+
+# Application versions
+
+ARG ARGO_VERSION='3.0.0'


Suggested change

ARG ARGO_VERSION='3.0.0'

ARG ARGO_VERSION='3.7.0'

craddm · 2025-08-12T09:08:09Z

.devcontainer/docker-compose.yml

+    # links:
+    #   - k3s-server:kubernetes.default.svc.cluster.local
+#     networks:
+#       - devcontainer_network
+  # k3s-server:
+  #   image: "rancher/k3s:${K3S_VERSION:-latest}"
+  #   command:
+  #     - server
+  #     - "--flannel-backend=none" # Disable flannel (we use cilium)
+  #     - "--disable=traefik" # Disable traefik ingress controller
+  #   tmpfs:
+  #   - /run
+  #   - /var/run
+  #   ulimits:
+  #     nproc: 65535
+  #     nofile:
+  #       soft: 65535
+  #       hard: 65535
+  #   privileged: true
+  #   restart: always
+  #   environment:
+  #   - K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml
+  #   - K3S_KUBECONFIG_MODE=666
+  #   volumes:
+  #   - k3s-server:/var/lib/rancher/k3s
+  #   # This is just so that we get the kubeconfig file out
+  #   - k8s-config:/output
+  #   expose:
+  #     - 6443 # Kubernetes API Server
+  #     - 80 # Ingress controller port 80 - not used at the moment
+  #     - 443 # Ingress controller port 443 - not used at the moment
+  #     - 2746  # Argo-workflows UI
+
+  #   ports:
+  #   - 6443:6443  # Kubernetes API Server
+  #   # - 80:80      # Ingress controller port 80
+  #   # - 443:443    # Ingress controller port 443
+  #   - 2746:2746  # Argo-workflows UI


Can we get rid of all these commented lines?

craddm · 2025-08-12T09:29:19Z

.devcontainer/post-create.sh

+print_message "DevContainer setup complete! 🎉"
+echo "You can now use the following commands:"
+echo "run 'source ~/.zshrc'"
+echo "run 'k3d cluster create --config k3d-default.yaml && /workspace/infra/fridge/k3d-default.yaml && /workspace/scripts/Install-Cillium.sh' to create a new cluster"


Note: this doesn't work as there is no k3d-default.yaml.

Not sure what options were configured in the k3d-default.yaml - do we just need to disable Traefik or is there more?

ah yes I forgot to track the file in git

does this also maybe make more sense to be moved into .devcontainer?

craddm · 2025-08-12T09:29:39Z

.devcontainer/devcontainer.Dockerfile

+    fzf \
+    python3.12-venv
+
+# install yq


Why do we need yq?

useful for parsing json output which a lot of CLI tools let you output to

I do also realise that this can just be installed via apt so I'll at least change it to that

craddm · 2025-08-12T09:32:00Z

.devcontainer/devcontainer.json

+    "workspaceFolder": "/workspace",
+    "service": "devcontainer",
+    "features": {
+        "ghcr.io/devcontainers-extra/features/kubectl-asdf": {


We could keep it as kubectl-asdf or switch to the kubectl-helm-minikube one (appreciate that installs minikube, which we don't need); if we don't switch, we should install helm manually in the Dockerfile

craddm · 2025-08-12T09:34:00Z

infra/fridge/scripts/init-pulumi.py

I'd say delete this file or move it into the .devcontainer folder; but it shouldn't be here

craddm · 2025-08-12T09:34:36Z

infra/fridge/scripts/pulumi-setup.sh

Same comment here; happy for these go in the .devcontainer folder

craddm · 2025-08-12T09:37:50Z

scripts/Install-Cillium.sh

Again, rather see this in the .devcontainer folder. Could see it in the future going in an infra/local folder (like the infra/aks/ folder, which is meant to build the cluster on AKS)

craddm · 2025-08-12T09:49:05Z

.devcontainer/devcontainer.Dockerfile

Let's add installation of Helm if we're not doing it using a devcontainer feature

craddm · 2025-10-14T14:41:11Z

Closing this, as this proved unworkable for the development path we took

jakewatson-bristol requested a review from craddm July 30, 2025 15:17

Jake Watson added 9 commits July 31, 2025 15:38

Add dev container for local cluster and dev

18bd423

Add kind config to define k8s cluster

cc9004d

Add helper scripts and just command runner

0b61ae7

remove pulumi set up scripts

60d166a

remove old k3s cluster set up refs

0f6696a

Remove just and gum to keep devcontainer simple

33f3a74

Add k9s and Hubble cli

7944bdd

disable kubeproxy for kind (use cilium)

ea4ec9d

Add missing argo cli completions

a49704e

jakewatson-bristol force-pushed the devcontainer branch from f9cf32d to 38c2949 Compare August 11, 2025 13:51

Jake Watson and others added 9 commits August 11, 2025 14:26

Enable hubble relay for hubble UI

60c028c

Run Install-cilium.sh after cluster creation

330ee36

add k3d

a04e049

Add helper scripts and just command runner

e6b253c

white space fixes

ecf88b2

Add init py to fridge/components

ec2e3dc

Delete individual imports from components and add components module n…

680ea56

…ame before class names

Correct K8s environment in storage class component

8e8a960

Remove reference to deleted file

5d2f0ea

jakewatson-bristol force-pushed the devcontainer branch from 2dc0c28 to 5d2f0ea Compare August 11, 2025 14:33

craddm and others added 5 commits August 11, 2025 15:34

Merge branch 'main' into devcontainer

6cfe079

remove just command runner

d3a72bd

add .kube dir to git ignore

4915a74

use k3d for local cluster

4e95bbb

Include cillium install

d05bbfa

craddm reviewed Aug 12, 2025

View reviewed changes

craddm requested changes Aug 12, 2025

View reviewed changes

j-watson added 8 commits August 13, 2025 13:58

remove kind install

49aa4b5

fix typos and install using argo workflows with ARG

0878598

remove old comments

383951f

Move scripts to .devcontainer

48393fc

Install Helm

e7079c9

Install yq from apt

3d23fe0

changes to make tab completion work more consistently

26103bd

Use ARG to set tool versions

604c6b0

craddm closed this Oct 14, 2025

	RUN curl -sLO "https://github.com/argoproj/argo-workflows/releases/download/v3.7.0/argo-$ARGO_OS-amd64.gz"
	RUN curl -sLO "https://github.com/argoproj/argo-workflows/releases/download/$ARGO_VERSION/argo-$ARGO_OS-amd64.gz"

Devcontainer for local dev #93

Devcontainer for local dev #93

Uh oh!

Conversation

jakewatson-bristol commented Jul 28, 2025

Uh oh!

craddm commented Jul 30, 2025

Uh oh!

awalford16 commented Aug 8, 2025

Uh oh!

craddm commented Aug 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

craddm commented Oct 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants