Skip to content

sgx-psw+aesmd: 2.25 -> 2.27#489368

Open
phlip9 wants to merge 5 commits intoNixOS:masterfrom
phlip9:sgx-psw-2.27
Open

sgx-psw+aesmd: 2.25 -> 2.27#489368
phlip9 wants to merge 5 commits intoNixOS:masterfrom
phlip9:sgx-psw-2.27

Conversation

@phlip9
Copy link
Contributor

@phlip9 phlip9 commented Feb 11, 2026

Changes

Update the sgx-psw package to the latest 2.27 release.

Fix the aesmd service, which broke with the update to systemd-v257 (#356818).

Update the sgx-azure-dcap-client to 1.13.0 (pre-release), which uses the new v4 Intel PCCS API as the prior v2 and v3 API versions are EOL on 2026-04-30.

If possible, it would be nice to backport this to release-25.11, as sgx-psw/aesmd are currently broken on that release.

Previous update PRs:

Quick Glossary:

  • Intel SGX is a Confidential Computing technology for running hardware-isolated enclaves and confidential VMs (Intel TDX) on Intel server CPUs alongside an untrusted hypervisor/Linux/userspace software stack.
  • sgx-psw (Platform SoftWare) provides the aesmd service (Architecture Enclave Service Manager Daemon), which simplifies running enclaves and getting remote attestation quotes.

Testing:

These changes were tested on an SGX-enabled Azure gen2 VM (DCSv3) running NixOS.

Run against real SGX hardware

Make sure you're running on a recent x86-64 Intel CPU, against a somewhat recent kernel with the in-tree kernel SGX driver (any NixOS config in the last few years should cover this).

Check the hardware and kernel setup:

$ journalctl --boot --dmesg --grep=sgx
kernel: sgx: EPC section 0x2c0000000-0x3bfffffff

In your NixOS configuration.nix, add something like:

  services.aesmd = {
    enable = true;
    # Include this if you're running on Azure
    quoteProviderLibrary = pkgs.sgx-azure-dcap-client;
  };

After a nixos-rebuild switch, check that the devices are configured and the aesmd service is running:

$ find /dev -name "*sgx*" -ls
 83      0 crw-rw----   1 root     sgx_prv   10, 126 May  8 17:21 /dev/sgx_provision
 84      0 crw-rw----   1 root     sgx       10, 125 May  8 17:21 /dev/sgx_enclave
400      0 drwxr-xr-x   2 root     root           80 May  8 17:21 /dev/sgx

$ systemctl status aesmd.service
● aesmd.service - Intel Architectural Enclave Service Manager
     Loaded: loaded (/etc/systemd/system/aesmd.service; enabled; preset: ignored)
     Active: active (running) since Wed 2026-02-11 08:52:20 UTC; 30min ago
 Invocation: 57a4f266b02b42c0a1d26cf3e7b3e8fa
    Process: 2170 ExecStartPre=/nix/store/d5p50m46spz9074r84yj4pmilm4x1z3g-copy-aesmd-data-files.sh (code=exited, status=0/SUCCESS)
   Main PID: 2185 (aesm_service)
         IP: 16.9K in, 3K out
         IO: 0B read, 16K written
      Tasks: 4 (limit: 38481)
     Memory: 4.5M (peak: 4.9M)
        CPU: 369ms
     CGroup: /system.slice/aesmd.service
             └─2185 /nix/store/fs5zwavky3hlq0bhwhgylxq7zcy1jf1q-sgx-psw-2.27.100.1/aesm/aesm_service --no-daemon

Feb 11 08:52:20 lexe-dev aesm_service[2185]: epid_quote_service_bundle_name:2.0.0
Feb 11 08:52:20 lexe-dev aesm_service[2185]: le_launch_service_bundle_name:2.0.0
Feb 11 08:52:20 lexe-dev aesm_service[2185]: linux_network_service_bundle_name:2.0.0
Feb 11 08:52:20 lexe-dev aesm_service[2185]: pce_service_bundle_name:2.0.0
Feb 11 08:52:20 lexe-dev aesm_service[2185]: quote_ex_service_bundle_name:2.0.0
Feb 11 08:52:20 lexe-dev aesm_service[2185]: system_bundle:4.0.0
Feb 11 08:52:20 lexe-dev aesm_service[2185]: Failed to set logging callback for the quote provider library.
Feb 11 08:52:20 lexe-dev aesm_service[2185]: The server sock is 0x55facb02fa40
Feb 11 08:53:01 lexe-dev aesm_service[2185]: InKernel LE loaded
Feb 11 08:53:01 lexe-dev aesm_service[2185]: Azure Quote Provider: libdcap_quoteprov.so [INFO]: Debug Logging Enabled

$ ls -l /var/run/aesmd/aesm.socket
srwxrwxrwx 1 aesmd sgx 0 Jun 26 00:19 /var/run/aesmd/aesm.socket

Run a test enclave that exercises remote attestation:

$ nix run -L github:lexe-app/lexe-public#run-sgx-test
Ensure SGX platform primitives work (sealing, attestation, etc)
machine_id: e6f7d35736746f205a71551771263b86
measurement: eac7ec70f2de593d368f4e56622a0cd9121299ea8d423a55d687c5fbaccb7444

SEALING
seal('label', 'my data') := 4c00000004000100000000000e0e100fffff010000000000000000000700000000000000e70000000000000056fb656bd71b051ba6a95a9a9e6f35a65954215f4cdbda2fae2ad4edff95ceff000000001700000018330baa813a2b8cb80a351935551f1ffb4f1da134714a

REMOTE ATTESTATION
fake pubkey we're attesting to: 4545454545454545454545454545454545454545454545454545454545454545
SGX DER-serialized evidence:
quote: 03000200000000000b00..4452d2d2d2d2d0a00

SGX enclave Report:
measurement: eac7ec70f2de593d368f4e56622a0cd9121299ea8d423a55d687c5fbaccb7444
mrsigner: 9affcfae47b848ec2caf1c49b4b283531e1cc425f93582b36806e52a43d78d1a
reportdata: 45454545454545454545454545454545454545454545454545454545454545450000000000000000000000000000000000000000000000000000000000000000
attributes: Attributes { flags: INIT | MODE64BIT, xfrm: 231 }
miscselect: (empty)
cpusvn: 10101110ffff01000000000000000000
isvsvn: 0
isvsvn: 0

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

@nixpkgs-ci nixpkgs-ci bot added 8.has: package (update) This PR updates a package to a newer version 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Feb 11, 2026
Comment on lines -75 to +76
tar -xzvf ${dcap.prebuilt} -C $sourceRoot/external/dcap_source ./prebuilt/
tar -xzvf ${dcap.prebuilt} -C $sourceRoot/external/dcap_source/QuoteGeneration ./psw/
tar -xzvf ${dcap.prebuilt} -C $sourceRoot/external/dcap_source prebuilt/
tar -xzvf ${dcap.prebuilt} -C $sourceRoot/external/dcap_source/QuoteGeneration psw/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason the paths changed slightly between releases and tar fails to extract unless the paths match exactly:

$ tar -tvf /nix/store/wwrz1bajkv340p7168fald0fhm7jghj9-prebuilt_dcap_1.24.tar.gz
drwxr-xr-x bgotowal/intelall 0 2025-10-29 17:07 prebuilt/
drwxr-xr-x bgotowal/intelall 0 2025-10-29 17:07 prebuilt/opa_bin/
-rw------- bgotowal/intelall 227664 2025-10-29 17:07 prebuilt/opa_bin/policy.wasm
drwxr-xr-x bgotowal/intelall      0 2025-10-29 00:44 prebuilt/openssl/
drwxr-xr-x bgotowal/intelall      0 2025-10-29 18:45 prebuilt/openssl/lib/
-rw-r--r-- bgotowal/intelall 9606146 2025-10-29 21:52 prebuilt/openssl/lib/libcrypto.a
drwxr-xr-x bgotowal/intelall       0 2025-10-29 00:41 prebuilt/openssl/lib/linux64/
-rw-r--r-- bgotowal/intelall 9606146 2025-10-29 21:59 prebuilt/openssl/lib/linux64/libcrypto.a
# ...

$ tar -tvf /nix/store/2mxs3avyf51rr8wd1l2lglwllpx43pjb-prebuilt_dcap_1.23.tar.gz
drwxrwxr-x feng/feng         0 2024-09-09 16:20 ./
drwxrwxr-x feng/feng         0 2024-07-10 16:29 ./prebuilt/
drwxrwxr-x feng/feng         0 2024-07-25 17:25 ./prebuilt/opa_bin/
-rw------- feng/feng    227664 2024-07-25 17:25 ./prebuilt/opa_bin/policy.wasm
drwxrwxr-x feng/feng         0 2024-07-10 16:21 ./prebuilt/openssl/
drwxrwxr-x feng/feng         0 2024-07-10 16:21 ./prebuilt/openssl/inc/
drwxrwxr-x feng/feng         0 2024-07-10 16:21 ./prebuilt/openssl/inc/openssl/
-rw-rw-r-- feng/feng     29352 2024-07-10 16:21 ./prebuilt/openssl/inc/openssl/ocsp.h
# ...

@phlip9
Copy link
Contributor Author

phlip9 commented Feb 20, 2026

@marcin-serwin Thanks for the review. All comments should be addressed.

Copy link
Contributor

@marcin-serwin marcin-serwin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I have no experience with running this so I'd prefer to have a second opinion before merging.

@nixpkgs-ci nixpkgs-ci bot added 2.status: merge conflict This PR has merge conflicts with the target branch 12.approvals: 1 This PR was reviewed and approved by one person. labels Feb 21, 2026
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Feb 21, 2026
@phlip9
Copy link
Contributor Author

phlip9 commented Feb 21, 2026

Needed to rebase on master after a repo-wide finalAttrs refactor commit touched a deleted file (pkgs/os-specific/linux/sgx/sdk/default.nix).

@phlip9
Copy link
Contributor Author

phlip9 commented Feb 21, 2026

cc @haraldh are you still using SGX at all? any chance you could give this a test real quick? Thanks!

@haraldh
Copy link
Contributor

haraldh commented Feb 21, 2026

sorry, only Sev-SNP and TDX

This package has been broken and unmaintained since 2024-11, when it
broke sometime around the nixos-24.11 release
@phlip9
Copy link
Contributor Author

phlip9 commented Feb 25, 2026

rebased on master

@phlip9
Copy link
Contributor Author

phlip9 commented Feb 27, 2026

Updated the aesmd service to fix an issue and cleaned up some old OOT+DCAP compat stuff.

On our prod machines, aesmd would always fail the first time after boot, with logs like:
XXX-copy-aesmd-data-files.sh[770]: chown: invalid user: 'aesmd:aesmd'.
Everything would be fine after the next restart, but blocking boot for 15 seconds is definitely annoying.

It appears the issue was that we were trying to name the DynamicUser User and Group (aesmd/aesmd) during ExecStartPre, but systemd only creates these right before ExecStart. After playing around with the ExecStartPre script, I finally got it to work correctly. As a bonus, we can also re-add the RootDirectory/chroot hardening.

I've also cleaned out the config for the old out-of-tree isgx driver and the old DCAP driver. The kernel has had in-kernel SGX support since 5.11 (~2021/02).

github-actions[bot]

This comment was marked as outdated.

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-nixos-tests This PR causes rebuilds for all NixOS tests and should normally target the staging branches. and removed 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. labels Feb 27, 2026
On our prod machines, aesmd would always fail the first time after boot,
with logs like:
`XXX-copy-aesmd-data-files.sh[770]: chown: invalid user: 'aesmd:aesmd'`.
Everything would be fine after the next restart, but blocking boot for
15 seconds is definitely annoying.

It appears the issue was that we were trying to name the DynamicUser
User and Group (aesmd/aesmd) during ExecStartPre, but systemd only
creates these right before ExecStart. After playing around with the
ExecStartPre script, I finally got it to work correctly. As a bonus, we
can also re-add the RootDirectory/chroot hardening.

I've also cleaned out the config for the old out-of-tree isgx driver and
the old DCAP driver. The kernel has had in-kernel SGX support since 5.11
(~2021/02).
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. and removed 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-nixos-tests This PR causes rebuilds for all NixOS tests and should normally target the staging branches. labels Feb 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (update) This PR updates a package to a newer version 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants