unable to sign with sign cli + Google Cloud KMS Provider

[sig-ogeorge]

Describe the bug
I am attempting to use sign cli with Google KMS CNG on Windows Server 2022. The sign command is throwing an exception.

Execute: sign code certificate-store --certificate-fingerprint a643e05776922b19a43d71c5c459e8a06bccc7a7f879a0a7ef9dd9d3390894e9 -cf self-signed.crt -t http://timestamp.digicert.com/ -v debug -td sha256 -fd sha256 -csp "Google Cloud KMS Provider" -k "projects/google_proj/locations/us-east5/keyRings/mykeyring/cryptoKeys/my-private-key/cryptoKeyVersions/1" test.vsix

### Result:
I1028 22:59:29.978845 9292 bridge.cc:165] OpenKey invoked
Provider: 1960203581840
Key name: projects/google_proj/locations/us-east5/keyRings/mykeyring/cryptoKeys/my-private-key/cryptoKeyVersions/1
LegacyKeySpec: 0
Flags: 64

I1028 22:59:29.980263 9292 logging.cc:81] returning 0x80090027 from OpenKeyFn due to status INVALID_ARGUMENT: at bridge.cc:187: unsupported legacy key spec specified: 0 [type.googleapis.com/kmscng.StatusDetails='SECURITY_STATUS=0x80090027']
I1028 22:59:29.980729 9292 bridge.cc:76] FreeProvider invoked
Provider: 1960203581840

fail: Sign.Core.ISigner[0]
The parameter is incorrect.
System.Security.Cryptography.CryptographicException: The parameter is incorrect.
at System.Security.Cryptography.CngKey.Open(String keyName, CngProvider provider, CngKeyOpenOptions openOptions)
at Sign.SignatureProviders.CertificateStore.CertificateStoreService.GetRsaAsync(CancellationToken cancellationToken) in //src/Sign.SignatureProviders.CertificateStore/CertificateStoreService.cs:line 84
at Sign.Core.AzureSignToolSigner.SignAsync(IEnumerable1 files, SignOptions options) in //src/Sign.Core/DataFormatSigners/AzureSignToolSigner.cs:line 95 at Sign.Core.AggregatingSigner.SignAsync(IEnumerable1 files, SignOptions options) in //src/Sign.Core/DataFormatSigners/AggregatingSigner.cs:line 204
at Sign.Core.AggregatingSigner.SignAsync(IEnumerable1 files, SignOptions options) in //src/Sign.Core/DataFormatSigners/AggregatingSigner.cs:line 92 at Sign.Core.Signer.<>c__DisplayClass3_0.<b__0>d.MoveNext() in //src/Sign.Core/Signer.cs:line 155 --- End of stack trace from previous location --- at System.Threading.Tasks.Parallel.<>c__531.<b__53_0>d.MoveNext()
--- End of stack trace from previous location ---
at Sign.Core.Signer.SignAsync(IReadOnlyList`1 inputFiles, String outputFile, FileInfo fileList, DirectoryInfo baseDirectory, String applicationName, String publisherName, String description, Uri descriptionUrl, Uri timestampUrl, Int32 maxConcurrency, HashAlgorithmName fileHashAlgorithm, HashAlgorithmName timestampHashAlgorithm) in //src/Sign.Core/Signer.cs:line 84

Repro steps

• gcloud kms keyrings create mykeyring --location=us-east5
• gcloud kms keys create my-private-key --keyring=mykeyring --project=google_proj --location=us-east5 --purpose="asymmetric-signing" --default-algorithm=***** --protection-level="hsm"
• curl -o libkmsp11.tar.gx https://github.com/GoogleCloudPlatform/kms-integrations/releases/download/pkcs11-v1.6/libkmsp11-1.6-linux-amd64.tar.gz
• tar -xf libkmsp11.tar.gz
• export PKCS11_MODULE_PATH=$(pwd)/libkmsp11-1.6-linux-amd64/libkmsp11.so
• export KMS_PKCS11_CONFIG=$(pwd)/pkcs11-config.yml
• openssl req -new -x509 -days 3650 -sha256 -engine pkcs11 -keyform engine -key pkcs11:object=my-private-key -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -subj '/CN=MyCorp/' -out self-signed.crt
• sign code certificate-store --certificate-fingerprint a643e05776922b19a43d71c5c459e8a06bccc7a7f879a0a7ef9dd9d3390894e9 -cf self-signed.crt -t http://timestamp.digicert.com/ -v debug -td sha256 -fd sha256 -csp "Google Cloud KMS Provider" -k "projects/google_proj/locations/us-east5/keyRings/mykeyring/cryptoKeys/my-private-key/cryptoKeyVersions/1" test.vsix

Expected behavior
I expocted completion with no exceptions or error codes

Actual behavior
info: Sign.Core.ISigner[0]
Submitting test.vsix for signing.
info: Sign.Core.ISigner[0]
SignAsync called for test.vsix. Using C:\cygwin64\tmp\oropyxa2.zhi\lidogpdh.vsix locally.
info: Sign.Core.IContainerProvider[0]
Extracting container C:\cygwin64\tmp\oropyxa2.zhi\lidogpdh.vsix to C:\cygwin64\tmp\5ef00hch.xf2.
info: Sign.Core.IDataFormatSigner[0]
Signing SignTool job with 253 files.
I1028 22:59:29.977450 9292 bridge.cc:52] OpenProvider invoked
Provider name: Google Cloud KMS Provider
Flags: 0

I1028 22:59:29.978845 9292 bridge.cc:165] OpenKey invoked
Provider: 1960203581840
Key name: projects/google_proj/locations/us-east5/keyRings/mykeyring/cryptoKeys/my-private-key/cryptoKeyVersions/1
LegacyKeySpec: 0
Flags: 64

I1028 22:59:29.980263 9292 logging.cc:81] returning 0x80090027 from OpenKeyFn due to status INVALID_ARGUMENT: at bridge.cc:187: unsupported legacy key spec specified: 0 [type.googleapis.com/kmscng.StatusDetails='SECURITY_STATUS=0x80090027']
I1028 22:59:29.980729 9292 bridge.cc:76] FreeProvider invoked
Provider: 1960203581840

fail: Sign.Core.ISigner[0]
The parameter is incorrect.
System.Security.Cryptography.CryptographicException: The parameter is incorrect.
at System.Security.Cryptography.CngKey.Open(String keyName, CngProvider provider, CngKeyOpenOptions openOptions)
at Sign.SignatureProviders.CertificateStore.CertificateStoreService.GetRsaAsync(CancellationToken cancellationToken) in //src/Sign.SignatureProviders.CertificateStore/CertificateStoreService.cs:line 84
at Sign.Core.AzureSignToolSigner.SignAsync(IEnumerable1 files, SignOptions options) in //src/Sign.Core/DataFormatSigners/AzureSignToolSigner.cs:line 95 at Sign.Core.AggregatingSigner.SignAsync(IEnumerable1 files, SignOptions options) in //src/Sign.Core/DataFormatSigners/AggregatingSigner.cs:line 204
at Sign.Core.AggregatingSigner.SignAsync(IEnumerable1 files, SignOptions options) in //src/Sign.Core/DataFormatSigners/AggregatingSigner.cs:line 92 at Sign.Core.Signer.<>c__DisplayClass3_0.<b__0>d.MoveNext() in //src/Sign.Core/Signer.cs:line 155 --- End of stack trace from previous location --- at System.Threading.Tasks.Parallel.<>c__531.<b__53_0>d.MoveNext()
--- End of stack trace from previous location ---
at Sign.Core.Signer.SignAsync(IReadOnlyList`1 inputFiles, String outputFile, FileInfo fileList, DirectoryInfo baseDirectory, String applicationName, String publisherName, String description, Uri descriptionUrl, Uri timestampUrl, Int32 maxConcurrency, HashAlgorithmName fileHashAlgorithm, HashAlgorithmName timestampHashAlgorithm) in //src/Sign.Core/Signer.cs:line 84

Additional context
sign --version
0.9.1-beta.24469.1+5516908609085978174904945d54144c117041dc

dotnet --info
.NET SDK:
Version: 8.0.403
Commit: c64aa40a71
Workload version: 8.0.400-manifests.18f19b92
MSBuild version: 17.11.9+a69bbaaf5

Runtime Environment:
OS Name: Windows
OS Version: 10.0.20348
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.403\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
There are no installed workloads to display.

Host:
Version: 8.0.10
Architecture: x64
Commit: 81cabf2857

.NET SDKs installed:
8.0.403 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
None

Environment variables:
Not set

global.json file:
Not found

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

[tdbhacks]

As an initial guess, I see rsa_padding_mode:pss and other RSA-PSS arguments in the repro steps, which makes me think you are using an RSA-PSS key, right? If that's the case, note that we only support a select list of algorithms in our provider. Contributions are always welcome of course, we've had one in the past for RSA-PKCS1: #29

However, I think this is failing even earlier at OpenKey, because the passed key spec is unexpected (should be AT_SIGNATURE, see https://learn.microsoft.com/en-us/windows/win32/api/ncrypt/nf-ncrypt-ncryptopenkey#parameters). It's unclear to me why the sign CLI would pass a value of 0 for dwKeyLegacySpec, but we've seen all sorts of weird/incorrect behaviors in the past..

Are you using signtool or another sign CLI? The sign command looks slightly different at first glance