az network bastion ssh + AAD fails with "WARNING: UNPROTECTED PRIVATE KEY FILE!"

[ckittel]

Describe the bug

When using az network bastion ssh --auth-type AAD the SSL certificate is automatically added to /tmp/aadsshcert... but the file permissions are set such that OpenSSH 8.2 fails with

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/tmp/aadsshcert23cd5o7q/id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

which means the connection cannot be established.

I know the bastion command delegates this work to the ssh extension, so this is probably more of an issue for the ssh extension authors to address.

Related command

az network bastion ssh --auth-type AAD

Errors

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/tmp/aadsshcert23cd5o7q/id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

Issue script & Debug output

cli.azext_ssh.ssh_utils: Running ssh-keygen command ssh-keygen -f /tmp/aadsshcert23cd5o7q/id_rsa -t rsa -q -N 
...
cli.azext_bastion.custom: Running ssh command /usr/bin/ssh [email protected]@localhost -i /tmp/aadsshcert23cd5o7q/id_rsa -o CertificateFile=/tmp/aadsshcert23cd5o7q/id_rsa.pub-aadcert.pub -p 34457 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=Error

Expected behavior

The generated id_rsa file is set to permissions of 600.

As a bonus, I think it could even to set to 400 since this directory is transient by nature. The temp directory (aadsshcert23cd5o7q in this specific example above) could even be set to 700 as well for added security/intent hygiene.

Environment Summary

azure-cli                         2.49.0

core                              2.49.0
telemetry                          1.0.8

Extensions:
bastion                            0.2.4
resource-graph                     2.1.0
ssh                                1.1.6

Dependencies:
msal                              1.20.0
azure-mgmt-resource               22.0.0

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @aznetsuppgithub.

Issue Details

---

Describe the bug

When using az network bastion ssh --auth-type AAD the SSL certificate is automatically added to /tmp/aadsshcert... but the file permissions are set such that OpenSSH 8.2 fails with

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/tmp/aadsshcert23cd5o7q/id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

which means the connection cannot be established.

I know the bastion command delegates this work to the ssh extension, so this is probably more of an issue for the ssh extension authors to address.

Related command

az network bastion ssh --auth-type AAD

Errors

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/tmp/aadsshcert23cd5o7q/id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

Issue script & Debug output

cli.azext_ssh.ssh_utils: Running ssh-keygen command ssh-keygen -f /tmp/aadsshcert23cd5o7q/id_rsa -t rsa -q -N 
...
cli.azext_bastion.custom: Running ssh command /usr/bin/ssh [email protected]@localhost -i /tmp/aadsshcert23cd5o7q/id_rsa -o CertificateFile=/tmp/aadsshcert23cd5o7q/id_rsa.pub-aadcert.pub -p 34457 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=Error

Expected behavior

The generated id_rsa file is set to permissions of 600.

As a bonus, I think it could even to set to 400 since this directory is transient by nature. The temp directory (aadsshcert23cd5o7q in this specific example above) could even be set to 700 as well for added security/intent hygiene.

Environment Summary

azure-cli                         2.49.0

core                              2.49.0
telemetry                          1.0.8

Extensions:
bastion                            0.2.4
resource-graph                     2.1.0
ssh                                1.1.6

Dependencies:
msal                              1.20.0
azure-mgmt-resource               22.0.0

Additional context

No response

Author:	ckittel
Assignees:	-
Labels:	bug, Network, Service Attention, needs-team-attention, Network - Bastion, Auto-Assign, Azure CLI Team
Milestone:	Backlog

[navba-MSFT]

Non customer reported. Adding Service team to look into this.

[ckittel]

@navba-MSFT how's this coming along?

[ckittel]

Any updates to share? We are introducing a pretty significant workaround due to this in our customer-facing content -- would be great to simplify this back to a process that "just works" based on how it was designed to operate.

[ckittel]

What's the decision here, @navba-MSFT? Any more information I can provide. Reach out to me on teams if you require a sync call.

[navba-MSFT]

@ckittel As mentioned above, this needs to be looked by Service team. Please follow-up with @aznetsuppgithub.

@aznetsuppgithub Please look into this on priority and reach out to @ckittel if you need any more information on this.

[isamorris]

@ckittel reviewing now

[earlb-cr2]

I faced the same issue when using command:
az network bastion ssh --name MyBastionName --resource-group MyGR --target-resource-id MyResourceID --auth-type AAD --debug

I modified line 348 in the following python script '%UserProfile%.azure\cliextensions\ssh\azext_ssh\custom.py'. To change the permission definition from '0o644' to '0o600' and this solved the permissions issue.
..
--oschmod.set_mode(cert_file, 0o644)
++oschmod.set_mode(cert_file, 0o600)
..

However, now I am faced with the dreaded 'invalid format' for the certificate file
Load key "%AppData%\Temp\aadsshxxxxx\id_rsa.pub-aadcert.pub": invalid format

Strangely, this error does not occur with the first generated certificate file, only subsequent generated files, even though files are identical.
Adding the exception to the sshd_config file does not help
PubkeyAcceptedAlgorithms [email protected]
CASignatureAlgorithms +ssh-rsa

When will we see support for ed25519 format?

[ckittel]

@isamorris

reviewing now

Hey there Isabelle, what did your review yield?

[n0rthdev]

@isamorris

are there any updates?

[ckittel]

Any updates @yonzhan or @isamorris?

[aavalang]

@vthiebaut10 can you take a look at this? This is likely a ssh extension issue as we can az ssh to generate the cert.

[ckittel]

Hey there @vthiebaut10, what did you find when you took a look at this?

[vthiebaut10]

Thank you for your patience. This problem is currently being investigated here Azure/azure-cli#28417

I'm currently out of office, and will continue the investigation as soon as I arrive. As a temporary workaround, the users in the other issue reported that downgrading Azure CLI fixes the issue.

[sciacca75]

Same issue for me

[vthiebaut10]

I shared a temporary workaround for this issue in the main issue: Azure/azure-cli#28417

[n0rthdev]

Thanks @vthiebaut10 for the workaround. This deeplink to the workaround mentioned above in Azure/azure-cli#28417 : Azure/azure-cli#28417