feat(deployment): FSS Deployment tool for Azure

[GeorgeDavis-TM]

FSS Deployment tool for Azure

Change Summary

• Introduction of a FSS deployment tool for Azure subscriptions
• The deployment tool support 3 modes of deployment -
  • geographies - 1 FSS Scanner Stack per Azure geographyGroup, Default
  • one-to-one - 1 FSS Scanner Stack per Storage Account (1:1 relationship)
  • single - Just 1 FSS Scanner Stack for all Storage Accounts (not recommended for multi-region storage accounts)

PR Checklist

• I've read and followed the Contributing Guide.
• Documents/Readmes
  • Updated accordingly
• Plugins that have versioning
  • Version bumped and follows Semantic Versioning

Other Notes

⚠️ Recommend to merge PR #87 #88 before merging this PR.

[trend-jack-c-tang]

Hi @GeorgeDavis-TM,
Thank you for this PR. This can help customers to protect lots of storage accounts at a time. I have few questions about the solution:

• How long does the function take to deploy stacks for, like 10, storage accounts? Consumption function app has 10 minutes as maximal timeout, and deploying one set of stacks would take that long time. Can the deployment run in parallel? Or can we use ARM to deploy that many stacks?
• One scanner stack for serving 50 storage stacks looks a bit aggressive to me. And some storage accounts that have higher uploading frequency would need dedicated scanner for each.
• How to monitor the failure and recover the process from failure when deploying multiple stacks?
• The way to configure credentials are not documented. Managed identity with a proper role would be another choice for you to refer to. By using it you won't need to configure credentials.

[GeorgeDavis-TM]

@jack-c-tang Thank you for your review. Please see my comments inline.

Hi @GeorgeDavis-TM, Thank you for this PR. This can help customers to protect lots of storage accounts at a time. I have few questions about the solution:

• How long does the function take to deploy stacks for, like 10, storage accounts? Consumption function app has 10 minutes as maximal timeout, and deploying one set of stacks would take that long time. Can the deployment run in parallel? Or can we use ARM to deploy that many stacks?

This tool was designed to run in a terminal or Azure Cloud Shell, with versions for Azure functions planned for future releases when we have achieved parallelity.

Please consider this tool, an MVP attempt at implementing deployment automation for FSS Stacks in Azure. I am looking for validation of this approach. Once validated from the product team, we can design for parallel runs.

• One scanner stack for serving 50 storage stacks looks a bit aggressive to me. And some storage accounts that have higher uploading frequency would need dedicated scanner for each.

The value is configurable in the config.json file, under cloudone.max_storage_stack_per_scanner_stack. The default is set to 50 as a maximum guideline. We can add a warning at the start of execution on rightsizing for the workload.

• How to monitor the failure and recover the process from failure when deploying multiple stacks?

At the moment, failure of a deployment, throws an exception and exits the script. Complex handling logic can be introduced in later stages, where we are deploying multiple stacks in parallel.

• The way to configure credentials are not documented. Managed identity with a proper role would be another choice for you to refer to. By using it you won't need to configure credentials.

At the moment, I am using an Azure registered app and secret from the same, similar to tm-fss-management.

[trend-jack-c-tang]

Hi @GeorgeDavis-TM ,
Sorry that I've been busy and cannot fully review this huge and rich-functional PR. In prod team Q3 OKR we have this item in the secondary priority, and this PR could be a start point for building this. Thank you for submitting this PR.

Due to resources and priority, the prod team might not be able to fully validate this solution in a short time. I tried to understand how the plugin works, but some points are still unclear to me. The deploy.sh script looks like just to deploy a function app and doesn't trigger it, so I don't get how to trigger the function. And there's no serverless template in the PR, why is there serverless deploy -s dev command in the README? It looks like it's the function app who deploys the stacks not the script, so the timeout hard limit needs to be considered. Maybe you can use premium tier which has unlimited timeout.

Do you think a short demo video clip can help us understand it quickly? If you have run some tests, like the number of storage accounts you've protected by this plugin, that would also be helpful. Thanks.

[GeorgeDavis-TM]

Hi @GeorgeDavis-TM , Sorry that I've been busy and cannot fully review this huge and rich-functional PR. In prod team Q3 OKR we have this item in the secondary priority, and this PR could be a start point for building this. Thank you for submitting this PR.

Due to resources and priority, the prod team might not be able to fully validate this solution in a short time. I tried to understand how the plugin works, but some points are still unclear to me. The deploy.sh script looks like just to deploy a function app and doesn't trigger it, so I don't get how to trigger the function. And there's no serverless template in the PR, why is there serverless deploy -s dev command in the README? It looks like it's the function app who deploys the stacks not the script, so the timeout hard limit needs to be considered. Maybe you can use premium tier which has unlimited timeout.

Do you think a short demo video clip can help us understand it quickly? If you have run some tests, like the number of storage accounts you've protected by this plugin, that would also be helpful. Thanks.

I will work on updating the README and get back to you for review. Thank you for your feedback.