Using docker hsm image

[avanide]

Hi,
I've noticed there are two docker images: standard one and an HSM one: https://hub.docker.com/r/smallstep/step-ca
We would like to use the HSM one and use a Yubikey plugged on the host machine. Each of our containers must follow the least privilege principle and we are not able to find out whether some capabilities are required to use this HSM image.

I've noticed on the helm chart, in the default values (https://artifacthub.io/packages/helm/smallstep/step-certificates?modal=values), that the security contexts are restricted:

• allowPrivilegeEscalation: false
• readOnlyRootFilesystem: true
• runAsUser: 1000
• All capabilities are dropped except NET_BIND_SERVICE
  Is it accurate only for the standard docker image or is it true for the HSM one too?

May you confirm which privileges are required to run the HSM docker image? How do we grant access to the yubikey plugged on the host?

Thanks,
Alex

[tashian]

I haven't tested this, but I think you'll have to figure out what USB device the YubiKey is using, and then grant that device to the container—eg, using docker run ... --device=/dev/ttyUSB0.

[avanide]

Hi Tashian,
Thanks for the answer. I understand it should work this way but I would expect to have clear confirmation. At least, my guess is that if there's such a docker image dedicated to HSM, somebody tried it (at least in the dev team).
Would be great to have official guidance from the product/dev team.

[tashian]

The difficulty of testing this is that it's vendor-specific.

• For example, with a Yubikey you may need to pass the USB device through.

• With a TPM, you'll need to run tpm2-abrmd(https://github.com/tpm2-software/tpm2-abrmd) on the host system and pass /dev/tpmrm0 through to the container.

• With a PKCS#11 HSM, you'd want to run the HSM PKCS11 broker service on the host and then make sure that port on localhost is accessible from within the container, and the HSM PKCS11 library is available in the container.

We're a small team and we can't test every scenario.
So, you'll have to try it out and see.
And if you learn something, please add your advice to the thread!

[danthonywalker]

I am also having trouble running the HSM image. When I run lsusb I can clearly see the YubiKey.

Bus 001 Device 001: ID 1d6b:0002 Linux 6.6.49-0-rpi xhci-hcd xHCI Host Controller
Bus 001 Device 002: ID 2109:3431  USB2.0 Hub
Bus 001 Device 003: ID 1050:0407 Yubico YubiKey OTP+FIDO+CCID
Bus 002 Device 001: ID 1d6b:0003 Linux 6.6.49-0-rpi xhci-hcd xHCI Host Controller

However, with the following docker-compose, I am getting the "error detecting yubikey: try removing and reconnecting the device" error.

services:
  stepca:
    image: smallstep/step-ca@hsm
    container_name: stepca
    restart: always
    stop_grace_period: 1m
    devices:
      - /dev/bus/usb:/dev/bus/usb
    ports:
      - 9000:9000
    volumes:
      - ./data:/home/step

networks:
  default:
    name: stepca

In the Discord, I see a few other posts with the same issue. One person was able to get it working by running the container as root. I tried with user: root and user: "0" but I still have this issue (maybe because I am running Docker in rootless mode?).

I was really hoping this would be simple to setup, but I am out of options on getting this to run.

Here are links to those Discord posts, in case it is helpful:
https://discord.com/channels/837031272227930163/841249977699401759/1186921025892515940
https://discord.com/channels/837031272227930163/841249977699401759/1132938661470675044

[tashian]

Hmm. I'd suggest trying to get ykman to run inside a container first, because that will simplify your testing. If ykman works, then try step-ca.

See also: https://developers.yubico.com/yubikey-manager/Device_Permissions.html

[Wakeful-Cloud]

I was able to get the HSM image working with a YubiKey on a K3s cluster with Akri without root. The main issue I ran into is that CRI-O ignores device permissions (For me, the device was owned by root:root and had 600 permissions in the container). I suspect other container runtimes may face similar issues, however. The workaround for this was to create a udev rule on the host like the below (in /etc/udev/rules.d/70-yubikey.rules):

SUBSYSTEMS=="usb", ATTRS{idVendor}=="YOUR_VID", ATTRS{idProduct}=="YOUR_PID", OWNER="1000", GROUP="1000", MODE="600"

A few notes on this udev rule:

1. You can obtain the VID and PID from lsusb or similar.
2. This works because, at least at the time of writing, the ownership/permission of the device on the host is the same as the ownership/permission of the device in the container. Specifically, the rule makes the device readable and writable by UID/GID 1000 (both in the container and on the host - you have been warned!).
3. UID/GID 1000 corresponds to the step user.

Then run sudo udevadm control --reload and sudo udevadm trigger to make the rule take effect. Once that's done, create an Akri configuration like:

apiVersion: akri.sh/v0
kind: Configuration
metadata:
  name: yubikey
spec:
  capacity: 1
  discoveryHandler:
    name: udev
    discoveryDetails: |
      groupRecursive: true
      udevRules:
        - ATTRS{idVendor}=="YOUR_VID", ATTRS{idProduct}=="YOUR_PID"
      permissions: rwm

and stateful set:

apiVersion: apps/v1
kind: StatefulSet
# ...
spec:
  # ...
  template:
    # ...
    spec:
      securityContext:
        fsGroup: 1000
      containers:
        - name: step-ca
          image: smallstep/step-ca:hsm
          securityContext:
            capabilities:
              add:
                - NET_BIND_SERVICE
              drop:
                - ALL
            seccompProfile:
              type: RuntimeDefault
            allowPrivilegeEscalation: false
            runAsNonRoot: true
            runAsUser: 1000
            runAsGroup: 1000
          command:
            - /bin/sh
            - -c
            - |
              # Start the PC/SC daemon
              pcscd
              
              # Start Smallstep CA
              exec /usr/local/bin/step-ca /path/to/your/config/ca.json
          resources:
            requests:
              akri.sh/yubikey-ID_HERE: "1" # Get the actual ID from kubectl get instances.akri.sh
            limits:
              akri.sh/yubikey-ID_HERE: "1" # Get the actual ID from kubectl get instances.akri.sh
          # ...