Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azurerm_databricks_workspace: managed service/disk support managed hsm key #27849

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

wuxu92
Copy link
Contributor

@wuxu92 wuxu92 commented Nov 1, 2024

Community Note

  • Please vote on this PR by adding a 👍 reaction to the original PR to help the community and maintainers prioritize for review
  • Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for PR followers and do not help prioritize for review

Description

DRAFT FOR INTERNAL REVIEW

This PR adds Managed HSM Key support for both managed services and managed disks.

It also deprecates the optional key vault ID property for both, as it is unnecessary. The Databricks server can validate the key's availability, so managed_services_cmk_key_vault_id and managed_disk_cmk_key_vault_id have been removed from the documentation. A deprecated description has been added to them, but they are not being removed from the schema to avoid introducing a breaking change.

PR Checklist

  • I have followed the guidelines in our Contributing Documentation.
  • I have checked to ensure there aren't other open Pull Requests for the same update/change.
  • I have checked if my changes close any open issues. If so please include appropriate closing keywords below.
  • I have updated/added Documentation as required written in a helpful and kind way to assist users that may be unfamiliar with the resource / data source.
  • I have used a meaningful PR title to help maintainers and other users understand this change and help prevent duplicate work.
    For example: “resource_name_here - description of change e.g. adding property new_property_name_here

Changes to existing Resource / Data Source

  • I have added an explanation of what my changes do and why I'd like you to include them (This may be covered by linking to an issue above, but may benefit from additional explanation).
  • I have written new tests for my resource or datasource changes & updated any relevent documentation.
  • I have successfully run tests with my changes locally. If not, please provide details on testing challenges that prevented you running the tests.
  • (For changes that include a state migration only). I have manually tested the migration path between relevant versions of the provider.

Testing

  • My submission includes Test coverage as described in the Contribution Guide and the tests pass. (if this is not possible for any reason, please include details of why you did or could not add test coverage)

image

Change Log

Below please provide what should go into the changelog (if anything) conforming to the Changelog Format documented here.

This is a (please select all that apply):

  • Bug Fix
  • New Feature (ie adding a service, resource, or data source)
  • Enhancement
  • Breaking Change

Related Issue(s)

Fixes #27738

Note

If this PR changes meaningfully during the course of review please update the title and description as required.

@wuxu92 wuxu92 force-pushed the databricks/cmk-hsm branch 2 times, most recently from e966538 to 3f24aa5 Compare November 1, 2024 03:21
@wuxu92 wuxu92 force-pushed the databricks/cmk-hsm branch from 904ccf6 to bd0c744 Compare November 1, 2024 14:25
@wuxu92 wuxu92 marked this pull request as ready for review November 1, 2024 14:30
@wuxu92 wuxu92 requested review from katbyte and a team as code owners November 1, 2024 14:30
@wuxu92 wuxu92 changed the title azurerm_databricks_workspace: managed service support managed hsm key azurerm_databricks_workspace: managed service/disk support managed hsm key Nov 2, 2024
@wuxu92 wuxu92 marked this pull request as draft November 3, 2024 02:01
Copy link
Collaborator

@WodansSon WodansSon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @wuxu92, I have given this PR and it mostly looks good, however I have left a few minor comments around the schema and depreciation text. If you can get those fixed up, I think this will be good to move out of draft state. 🚀

Comment on lines 295 to 299
"managed_services_cmk_key_vault_id": {
Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "Use 'managed_services_cmk_key_vault_key_id' instead, this field is not needed for cross subscription key vaults anymore",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this conflict with managed_services_cmk_key_vault_key_id?

Suggested change
"managed_services_cmk_key_vault_id": {
Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "Use 'managed_services_cmk_key_vault_key_id' instead, this field is not needed for cross subscription key vaults anymore",
"managed_services_cmk_key_vault_id": {
Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "'managed_services_cmk_key_vault_id' has been deprecated in favor of 'managed_services_cmk_key_vault_key_id' and will be removed in v5.0 of the provider",
ConflictsWith: []string{"managed_services_cmk_key_vault_key_id"},

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot add ConflictsWith here because some users may have set both managed_services_cmk_key_vault_id and managed_services_cmk_key_vault_key_id. Do you mean the new managed_disk_cmk_managed_hsm_key_id property should conflict with managed_services_cmk_key_vault_id? That would be reasonable. I updated it.

Comment on lines 315 to 320

"managed_disk_cmk_key_vault_id": {
Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "Use 'managed_disk_cmk_key_vault_key_id' instead, this field is not needed for cross subscription key vaults anymore",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"managed_disk_cmk_key_vault_id": {
Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "Use 'managed_disk_cmk_key_vault_key_id' instead, this field is not needed for cross subscription key vaults anymore",
"managed_disk_cmk_key_vault_id": {
Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "'managed_disk_cmk_key_vault_id' has been deprecated in favor of 'managed_disk_cmk_key_vault_key_id' and will be removed in v5.0 of the provider",
ConflictsWith: []string{"managed_disk_cmk_key_vault_key_id"},

Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: keyVaultValidate.KeyVaultChildID,
ConflictsWith: []string{"managed_disk_cmk_managed_hsm_key_id"},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this also conflict with managed_disk_cmk_key_vault_id?

Suggested change
ConflictsWith: []string{"managed_disk_cmk_managed_hsm_key_id"},
ConflictsWith: []string{"managed_disk_cmk_managed_hsm_key_id", "managed_disk_cmk_key_vault_id"},

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like above, we cannot have it conflict with managed_disk_cmk_key_vault_id here.

@wuxu92 wuxu92 force-pushed the databricks/cmk-hsm branch from eb448b5 to ff0df84 Compare November 15, 2024 04:12
@wuxu92 wuxu92 marked this pull request as ready for review November 15, 2024 06:21
@WodansSon
Copy link
Collaborator

@wuxu92, thanks for pushing those changes so quickly. This LGTM now! 🚀

Copy link
Member

@stephybun stephybun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @wuxu92, could you please follow the guidance in our breaking changes guide when deprecating properties? Once those changes have been made can you please also provide testing evidence in both 4.x and 5.0 mode. Thanks!

@wuxu92
Copy link
Contributor Author

wuxu92 commented Nov 15, 2024

Hi @stephybun, since those two properties are not needed for the business, they are just read from the config and set back to the state even in the original logic. Therefore, I merely marked them as deprecated. If you think it's still necessary, I can add the required work for a breaking change.

@stephybun
Copy link
Member

Yes, please still follow the guidance that I linked. This will ensure that the property is actually removed in 5.0 and we don't need to remember to remove it from the schema in the week of the major release. Make sure to update any affected tests as well as the upgrade guide too.

@wuxu92 wuxu92 force-pushed the databricks/cmk-hsm branch from b934160 to 2d6f936 Compare November 18, 2024 02:10
Copy link

This PR is being labeled as "stale" because it has not been updated for 30 or more days.

If this PR is still valid, please remove the "stale" label. If this PR is blocked, please add it to the "Blocked" milestone.

If you need some help completing this PR, please leave a comment letting us know. Thank you!

@stephybun
Copy link
Member

@wuxu92 I started a review on this a while ago, but since then the CreateUpdate method in this resource has been split. Could you please rebase this PR? I can continue the review once that's been done.

@wuxu92
Copy link
Contributor Author

wuxu92 commented Feb 5, 2025

@stephybun I rebased the main branch and updated the corresponding code.

image

…deprecate xx_key_vault_id for cross subscription support as not required

add deprecate in 5.0 beta
@wuxu92 wuxu92 force-pushed the databricks/cmk-hsm branch from 8b0333a to c4c39a7 Compare February 17, 2025 21:43
Copy link
Member

@stephybun stephybun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for rebasing @wuxu92.

In addition to the comments left in-line, can you please provide local testing evidence for the tests in 4.x as well as 5.0 mode?

Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "'managed_services_cmk_key_vault_id' has been deprecated in favor of 'managed_services_cmk_key_vault_key_id' and will be removed in v4.0 of the provider",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Deprecated: "'managed_services_cmk_key_vault_id' has been deprecated in favor of 'managed_services_cmk_key_vault_key_id' and will be removed in v4.0 of the provider",
Deprecated: "'managed_services_cmk_key_vault_id' has been deprecated in favor of 'managed_services_cmk_key_vault_key_id' and will be removed in v5.0 of the AzureRM provider",

Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateKeyVaultID,
Deprecated: "'managed_disk_cmk_key_vault_id' has been deprecated in favor of 'managed_disk_cmk_key_vault_key_id' and will be removed in v4.0 of the provider",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Deprecated: "'managed_disk_cmk_key_vault_id' has been deprecated in favor of 'managed_disk_cmk_key_vault_key_id' and will be removed in v4.0 of the provider",
Deprecated: "'managed_disk_cmk_key_vault_id' has been deprecated in favor of 'managed_disk_cmk_key_vault_key_id' and will be removed in v5.0 of the AzureRM provider",

Comment on lines 494 to 506
clientHub := meta.(*clients.Client)
client := clientHub.DataBricks.WorkspacesClient
acClient := clientHub.DataBricks.AccessConnectorClient
lbClient := clientHub.LoadBalancers.LoadBalancersClient
accountClient := clientHub.Account
subscriptionId := accountClient.SubscriptionId
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was no real need to change this and it introduces an inconsistent style around the client variable declarations in the provider, can you please leave these long hand? I'll make a note of this and see if we can look to publish a style guide for the provider

Suggested change
clientHub := meta.(*clients.Client)
client := clientHub.DataBricks.WorkspacesClient
acClient := clientHub.DataBricks.AccessConnectorClient
lbClient := clientHub.LoadBalancers.LoadBalancersClient
accountClient := clientHub.Account
subscriptionId := accountClient.SubscriptionId
client := meta.(*clients.Client).DataBricks.WorkspacesClient
acClient := meta.(*clients.Client).DataBricks.AccessConnectorClient
lbClient := meta.(*clients.Client).LoadBalancers.LoadBalancersClient
accountClient := meta.(*clients.Client).Account
subscriptionId := accountClient.SubscriptionId

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll adjust this style as you suggested. I just don't like repeating the code, but I'm fine with it to stay consistent with the provider.

Comment on lines +118 to +130
// VersionPtr returns a pointer to the version string if it exists, otherwise nil.
// This is useful for resources supporting versioned and versionless keys.
func (k *KeyVaultOrManagedHSMKey) VersionPtr() *string {
if k.KeyVaultKeyId != nil && k.KeyVaultKeyId.Version != "" {
return pointer.To(k.KeyVaultKeyId.Version)
}

if k.ManagedHSMKeyId != nil {
return pointer.To(k.ManagedHSMKeyId.KeyVersion)
}

return nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any usages of this yet, where do you foresee this being used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's not used for this resource. However, I have a task list of resources that need to add HSM key support during my investigation. Some of them will require this method, so I put it here initially..

image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed azurerm_log_analytics_cluster_customer_managed_key is on your list here, I have started work on this as part of #27433

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sreallymatt Thank you for letting me know! It hasn't started yet, so I'll skip it as per your works.

@@ -197,3 +241,9 @@ func FlattenKeyVaultOrManagedHSMID(id string, hsmEnv environments.Api) (*KeyVaul

return nil, fmt.Errorf("cannot parse given id to key vault key nor managed hsm key: %s", id)
}

func FlattenKeyVaultOrManagedHSMIDByComponents(baseUri, name, version string, hsmEnv environments.Api) (*KeyVaultOrManagedHSMKey, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be possible to determine the key type (key vault or managed hsm) based on the contents of the base URI. With this we can instantiate and return the correct object ID using the relevant NewSomeResourceID function and wouldn't need to call the FlattenKeyVaultOrManagedHSMID function anymore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated this method by creating an instance using the corresponding construction method. However, the NewHSMXXID function requires the HSM name instead of the baseUri(which is typically returned by the API). Therefore, the HSM ID part still needs to parse the ID format string.

})
}

func testAccDatabricksWorkspace_CmkManagedHSMDiskOnly(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func testAccDatabricksWorkspace_CmkManagedHSMDiskOnly(t *testing.T) {
func testAccDatabricksWorkspace_cmkManagedHSMDiskOnly(t *testing.T) {

})
}

func testAccDatabricksWorkspace_CmkManagedHSMDiskOnlyAltSubscription(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func testAccDatabricksWorkspace_CmkManagedHSMDiskOnlyAltSubscription(t *testing.T) {
func testAccDatabricksWorkspace_cmkManagedHSMDiskOnlyAltSubscription(t *testing.T) {

@@ -48,21 +48,15 @@ The following arguments are supported:

~> **Note:** Downgrading to a `trial sku` from a `standard` or `premium sku` will force a new resource to be created.

* `managed_services_cmk_key_vault_id` - (Optional) Resource ID of the Key Vault which contains the `managed_services_cmk_key_vault_key_id` key.
* `managed_services_cmk_key_vault_key_id` - (Optional) Customer managed encryption properties using a versioned Key Vaule Key ID for the Databricks Workspace managed resources(e.g. Notebooks and Artifacts).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `managed_services_cmk_key_vault_key_id` - (Optional) Customer managed encryption properties using a versioned Key Vaule Key ID for the Databricks Workspace managed resources(e.g. Notebooks and Artifacts).
* `managed_services_cmk_key_vault_key_id` - (Optional) Customer managed encryption properties using a versioned Key Vault Key ID for the Databricks Workspace managed resources e.g. Notebooks and Artifacts.


-> **Note:** The `managed_services_cmk_key_vault_id` field is only required if the Key Vault exists in a different subscription than the Databricks Workspace. If the `managed_services_cmk_key_vault_id` field is not specified it is assumed that the `managed_services_cmk_key_vault_key_id` is hosted in the same subscriptioin as the Databricks Workspace.
* `managed_services_cmk_managed_hsm_key_id` - (Optional) Customer managed encryption properties using a versioned Managed Hardware Security Key ID for the Databricks Workspace managed resources(e.g. Notebooks and Artifacts).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `managed_services_cmk_managed_hsm_key_id` - (Optional) Customer managed encryption properties using a versioned Managed Hardware Security Key ID for the Databricks Workspace managed resources(e.g. Notebooks and Artifacts).
* `managed_services_cmk_managed_hsm_key_id` - (Optional) Customer managed encryption properties using a versioned Managed Hardware Security Key ID for the Databricks Workspace managed resources e.g. Notebooks and Artifacts.


-> **Note:** If you are using multiple service principals to execute Terraform across subscriptions you will need to add an additional `azurerm_key_vault_access_policy` resource granting the service principal access to the key vault in that subscription.

* `managed_disk_cmk_key_vault_id` - (Optional) Resource ID of the Key Vault which contains the `managed_disk_cmk_key_vault_key_id` key.
* `managed_disk_cmk_key_vault_key_id` - (Optional) Customer managed encryption properties using a versioned Key Vaule Key ID for the Databricks Workspace managed disks.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `managed_disk_cmk_key_vault_key_id` - (Optional) Customer managed encryption properties using a versioned Key Vaule Key ID for the Databricks Workspace managed disks.
* `managed_disk_cmk_key_vault_key_id` - (Optional) Customer managed encryption properties using a versioned Key Vault Key ID for the Databricks Workspace managed disks.

@wuxu92
Copy link
Contributor Author

wuxu92 commented Feb 25, 2025

@stephybun

tests with 5.0 flag enabled, there is a new failure which is not related to this PR, I can fix it in a separate PR:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

azurerm_databricks_workspace - missing support for keys from a Managed HSM Key Vault
5 participants