usbip runner does not work as a security dongle for Heads

[mkopec]

My use case: Testing HOTP in Heads in QEMU without a physical dongle attached to the host system - see issue linuxboot/heads#1842 for more context

Currently when attempting to run OEM Factory Reset in Heads on QEMU, the process fails due to not detecting a security dongle, despite the dongle being listed in lsusb. I think this is because secrets-app is missing so HOTP isn't working, but there could be other issues too

[robin-nitrokey]

The secrets app should be enabled in the usbip runner. Can you access it from the host system?

What QEMU configuration do you use to pass through the USB device?

[macpijan]

Hi @robin-nitrokey

@mkopec I would appreciate it if you can put more details here, so we can move this discussion forward.

[tlaurion]

I'm also trying to understand what should be passed to qemu here.

Maybe https://github.com/Nitrokey/nitrokey-3-firmware/blob/main/docs%2Fusbip.md should be clearer and be augmented on what is expected to be passed to qemu as usb device?

For example for Heads qemu targets, make call typically looks like this to pass real nk3 device when translated:

---

Ie, from Heads master:

• ./docker_repro.sh make BOARD=qemu-coreboot-whiptail-tpm2-hotp USB_TOKEN=Nitrokey3NFC
• ./docker_repro.sh make BOARD=qemu-coreboot-whiptail-tpm2-hotp USB_TOKEN=Nitrokey3NFC run

expends to

!!!!!! Build starts !!!!!!
Makefile:270: warning: overriding recipe for target 'all'
Makefile:251: warning: ignoring old recipe for target 'all'
swtpm socket \
	--tpm2 \
	--tpmstate dir="/home/user/heads/build/x86/qemu-coreboot-whiptail-tpm2-hotp/vtpm" \
	--flags "startup-clear" \
	--terminate \
	--ctrl type=unixio,path="/home/user/heads/build/x86/qemu-coreboot-whiptail-tpm2-hotp/vtpm/sock" &
sleep 0.5
qemu-system-x86_64 -drive file="/home/user/heads/build/x86/qemu-coreboot-whiptail-tpm2-hotp/root.qcow2",if=virtio \
	--machine q35,accel=kvm:tcg \
	-rtc base=utc \
	-smp 1 \
	-vga std \
	-m "$(cat "/home/user/heads/build/x86/qemu-coreboot-whiptail-tpm2-hotp/memory")" \
	-serial stdio \
	--bios "/home/user/heads/build/x86/qemu-coreboot-whiptail-tpm2-hotp/heads-qemu-coreboot-whiptail-tpm2-hotp-v0.2.0-2440-gff307d4.rom" \
	-object rng-random,filename=/dev/urandom,id=rng0 \
	-device virtio-rng-pci,rng=rng0 \
	-netdev user,id=u1 -device e1000,netdev=u1 \
	-chardev socket,id=chrtpm,path="/home/user/heads/build/x86/qemu-coreboot-whiptail-tpm2-hotp/vtpm/sock" \
	-tpmdev emulator,id=tpm0,chardev=chrtpm \
	-device tpm-tis,tpmdev=tpm0 \
	-device qemu-xhci,id=usb \
	-device usb-tablet \
	-drive file="/home/user/heads/build/x86/qemu-coreboot-whiptail-tpm2-hotp/usb_fd.raw",if=none,id=usb-fd-drive,format=raw \
	-device usb-storage,bus=usb.0,drive=usb-fd-drive \
	-device usb-host,vendorid=8352,productid=17074 \

So what should we point to here after official docs' cargo run?

Goal would be to have USB_TOKEN=Nitrokey3NFC_emulated run and be able to test regressions/pr/security audits/etc of hotp_verification + nk3 firmware changes without requiring to flash physical nk3, which enforces firmware rollback prevention (so no revert to last release of nk3 firmware version possible).

Thanks

[robin-nitrokey]

Looks like there’s something missing from the qemu-system snippet (it ends with \). Anyway, -device usb-host,vendorid=8352,productid=17074 should to be the relevant flag. The usbip runner uses the same VID/PID 0x20a0/0x42b2 so I would expect it to work.

What exactly is the command that fails and the error message?

[robin-nitrokey]

nitrokey-hotp-verification uses CCID to communicate with the NK3. Per default, the USBIP runner only supports CTAPHID communication, see the “Limitations” section in the USBIP guide. So even if the USB device itself shows up, it cannot be accessed via CCID. You can try using --features ccid as described in the guide. Maybe it works with the limited set of commands sent by the verification tool. Just make sure that other services like GnuPG’s scdaemon or PCSC don’t try to talk to it.

[tlaurion]

Looks like there’s something missing from the qemu-system snippet (it ends with \). Anyway, -device usb-host,vendorid=8352,productid=17074 should to be the relevant flag. The usbip runner uses the same VID/PID 0x20a0/0x42b2 so I would expect it to work.

What exactly is the command that fails and the error message?

I have no error message. The snippet above was qemu interfacing with physical nk3. I have not tried the usb runner yet.

[tlaurion]

nitrokey-hotp-verification uses CCID to communicate with the NK3. Per default, the USBIP runner only supports CTAPHID communication, see the “Limitations” section in the USBIP guide. So even if the USB device itself shows up, it cannot be accessed via CCID. You can try using --features ccid as described in the guide. Maybe it works with the limited set of commands sent by the verification tool. Just make sure that other services like GnuPG’s scdaemon or PCSC don’t try to talk to it.

My question was: was this tested with qemu?
If you say "try" and "might", I expect it wasn't tested?

Can someone from Nitrokey clone Heads master and try to make this work replicating above commands and report what is missing here from direct experience from Heads master?

Thanks

[tlaurion]

nitrokey-hotp-verification uses CCID to communicate with the NK3. Per default, the USBIP runner only supports CTAPHID communication, see the “Limitations” section in the USBIP guide. So even if the USB device itself shows up, it cannot be accessed via CCID. You can try using --features ccid as described in the guide. Maybe it works with the limited set of commands sent by the verification tool. Just make sure that other services like GnuPG’s scdaemon or PCSC don’t try to talk to it.

Excellent. Could give it a try:

Ie, from Heads master:

• ./docker_repro.sh make BOARD=qemu-coreboot-whiptail-tpm2-hotp USB_TOKEN=Nitrokey3NFC
• `./docker_repro.sh make BOARD=qemu-coreboot-whiptail-tpm2-hotp USB_TOKEN=Nitrokey3NFC run

But it seems like refered doc limitation section's bug #261 is blocker to this, referring to this discussion (no fix) : https://lore.kernel.org/lkml/ZBHxfUX60EyCMw5l@Sun/

[robin-nitrokey]

If you say "try" and "might", I expect it wasn't tested?

Correct, I’m not aware of anybody using the USBIP runner with qemu so far.

But it seems like refered doc limitation section's bug #261 is blocker to this

The bug does not occur on every command. Maybe this particular setup does not trigger the bug.

[tlaurion]

@robin-nitrokey . Can't build usb runner without following down rabbit in rabbit hole

ie:

• requirements for compilling https://github.com/Nitrokey/nitrokey-3-firmware/blob/main/docs%2Fusbip.md#requirements : unclear, going to main docs:
  • https://github.com/Nitrokey/nitrokey-3-firmware/tree/main?tab=readme-ov-file#dependencies uclear, pointing to
  • https://github.com/Nitrokey/nitrokey-3-firmware/blob/main/rust-toolchain.toml
    • Ok...
      • debian-12:
        • sudo apt install cargo clang
        • git clone https://github.com/Nitrokey/nitrokey-3-firmware.git
        • going back to https://github.com/Nitrokey/nitrokey-3-firmware/blob/main/docs%2Fusbip.md#running-the-simulation
          error: Package embedded-runner-lib v1.8.0-rc.1 (/home/user/nitrokey-3-firmware/runners/embedded)does not have featurelittlefs2-sys. It has an optional dependency with that name, but that dependency uses the "dep:" syntax in the features table, so it does not have an implicit feature with that name

@robin-nitrokey : clearer instructions to provide so I do not spend hours trying to figure out what is missing in docs (not rust develoer here) to build usb-runner and test --features ccid, since from my understanding, qemu config as of today if device exposed on host running qemu should make same producit discovered and should give results to advance this ticket? Thanks

[robin-nitrokey]

@tlaurion Do you happen to use Docker or Nix? Then I could directly provide you a working environment.

since from my understanding, qemu config as of today if device exposed on host running qemu should make same producit discovered and should give results to advance this ticket?

Yes. If you have a way to list USB devices, it should already show up when using the pre-compiled usbip runner e. g. for v1.8.0-rc.1 (usbip-runner-v1.8.0-rc.1), it just won’t be recognized by the tooling expecting a CCID device.

[tlaurion]

@tlaurion Do you happen to use Docker or Nix? Then I could directly provide you a working environment.

since from my understanding, qemu config as of today if device exposed on host running qemu should make same producit discovered and should give results to advance this ticket?

Yes. If you have a way to list USB devices, it should already show up when using the pre-compiled usbip runner e. g. for v1.8.0-rc.1 (usbip-runner-v1.8.0-rc.1), it just won’t be recognized by the tooling expecting a CCID device.

The goal would be to pack requirements under Heads flake.nix, create a modules/nk3-usb-runner and have Heads qemu-coreboot-fbwhitpail-tpm2-hotp board as in example be able to use either usb physical or emulated nk3 in test environement.

Clear dependencies and build steps needed as per prior comment. So if you have flake.nix declared dependencies to point to, yes it would help since those would be added into Heads docker image built from nix develop environement. That would be ideal, since as noted, docs are not clear enough to be followed to build on top of debian-12 and I would prefer to not have to document this if it can be shipped with next revision of docker image and be able to document as code instead.

Thanks

[robin-nitrokey]

I don’t have a flake but a shell.nix:

{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/da4024d0ead5d7820f6bd15147d3fe2a0c0cec73.tar.gz") {} }:

let
  fenix = pkgs.callPackage
    (pkgs.fetchFromGitHub {
      owner = "nix-community";
      repo = "fenix";
      rev = "1a92c6d75963fd594116913c23041da48ed9e020";
      hash = "sha256-L3vZfifHmog7sJvzXk8qiKISkpyltb+GaThqMJ7PU9Y=";
    })
    { };
  toolchain = fenix.fromToolchainFile {
    dir = ./.;
  };
in
pkgs.mkShell {
  nativeBuildInputs = with pkgs.buildPackages; [
    clang
    git
    gnumake
    libclang
    toolchain
  ];

  shellHook = ''
    export LIBCLANG_PATH="${pkgs.libclang.lib}/lib"
  '';
}

If you place it in the root directory of the repository, you can run:

nix-shell --pure --run "cargo run --manifest-path runners/usbip/Cargo.toml --features ccid"

This contains the same dependencies as listed in the usbip guide. But Debian stable does not package the required Rust version. The easiest way would probably be to install rustup manually.

[tlaurion]

Was not able to easily add this under flake, will test after feature freeze it seems. If someone wants this for feature freeze, help needed.