publish bicep modules to ACR with azure devops

[JeroenvdBurg]

Hi all,

with the new private registry functionality I have a question about updating modules in the registry.

Let's say you have github repo with some modules witch you publish to an Azure container registry. You can do this manually with the bicep publish command, but if you want to automate this with an release pipeline in devops it becomes somewhat complicated.

How can you automaticly update the tag if a module file has changed?

I have now created a script witch performs the Az publish commands. But you have to manually upgrade the tag in the script.
If you could specify the tag/version in the module file witch would be used for publishing it would be easier because then you can just publish all the modules.

Any thoughts or help on this topic would be really helpfull. Thanks in advance.

[alex-frankel]

But you have to manually upgrade the tag in the script.

You can use environment variables or script arguments to make the script dynamic. Then you will need a way of generating a version and providing that to the script. You might want to look at something like NerdBank.

[thepaulmacca]

The plan is to use NerdBank.GitVersioning and their monorepo capabilities. Each module directory will have a version file that sets the major and minor version part (4.2, for example) then the number of commits since the last commit that touched the version file is used as the revision part of the module version. If there were 13 commits since the version file was touched, the version will be 4.2.13. Next commit will make it 4.2.14 and so on.

Each module in the public registry will be versioned independently. In other words, there won't be a shared repo-side module version.

It would be great to see some kind of demo on this in the next community call, for people looking to replicate a similar setup in their private registries

Would this be possible?

[cloudfright]

@thepaulmacca - it looks like path filters is the way to achieve this module folder level versioning. If Nerdbank handles the version increments based on commits and then a deployment script compares the module version number to the versions the Bicep ACR, it could be a neat way of only pushing changed modules to the registry.

[thepaulmacca]

Yeah that looks like a good shout. I'd love to see a CI setup on Azure Pipelines or something, to see how it would all tie together

[alex-frankel]

Once the public registry is available, we will try to have all our CI workflows in GH Actions easily shareable such that those that want to run their own registry like the "official" public registry can do so easily. It will probably take some time to work out the kinks, but that is our goal.

[thepaulmacca]

Once the public registry is available, we will try to have all our CI workflows in GH Actions easily shareable such that those that want to run their own registry like the "official" public registry can do so easily. It will probably take some time to work out the kinks, but that is our goal.

That would be amazing Alex, thanks for the update!

[Agazoth]

@JeroenvdBurg We are documenting our modules in a readme stored along with our modules. When updating the modules, we also need to update the readme to reflect the changes. When doing that, we document the version bump in the readme too.

On top of that, our pipeline evaluates, wether the readme version has been bumped or not. If the readme has been bumped, the pipeline uploads a new version, with the version number from the readme, to the registry. Both outcomes (bump or not) are recorded in the pipeline.

Prior to this, we kept a variable with a version number within the module, but the current approach gives us a cleaner module and endures, that we update the documentation.

[JeroenvdBurg]

Sounds like a solid work flow. Thanks for sharing this. I'm not sure which CI/CD pipeline you are using but if it is Azure Devops or Github actions would it be possible to share (a part) of the yml file?

[Agazoth]

Sure. We have an Azure DevOps pipeline, that does these steps (pipeline has been shortened):

steps:
  - pwsh: |
      curl -Lo bicep "$(bicepuri)"
      chmod +x ./bicep
      sudo mv ./bicep /usr/local/bin/bicep
      bicep -v
  - task: AzurePowerShell@5
      displayName: 'Upload module'
      inputs:
        azureSubscription: $(serviceconnection)
        ScriptType: InlineScript
        Inline: |
          $ModulePath = Join-Path $(pwd).Path 'scripts/Modules/ADOPipeline'
          Import-Module $ModulePath
          $Version = Get-VersionFromReadme $(ReadmeFile) -ReturnHighestProductionVersion
          $PublishedVersionTags = Get-AzContainerRegistryTag -RegistryName "$(acr)" -RepositoryName $('{0}/{1}' -f "$(env)",$Name) -ErrorAction SilentlyContinue
          $upload = $true
          if ("$(Build.SourceBranchName)" -eq "main" -and $PublishedVersionTags.Tags.Name -contains $Version){
            $upload = $false
            Write-Host "$Version already exists on $(acr). Update Readme.md to upload to main"
          }
          if ($upload){
            Write-Host "Uploading to $acrStr"
            bicep publish ($name + ".bicep") --target $acrStr
          }
        azurePowerShellVersion: 'latestVersion'
        pwsh: true

Our custom cmdlet Get-VersionFromReadme parses our readme and finds the table with the versions
Then we get the versions in the registry with Get-AzContainerRegistryTag
The we check if branch name is main. If it is, and already contains the version from the readme, we set upload to false.

We chose to keep everything in the pipeline, for devs to se what happens. Wrapping everything in a deployment script requires you to open the script in stead of looking in the pipeline, where you are working anyway.

[JeroenvdBurg]

thanks a lot , really appreciate this.

[lukaferlez]

Built my own scripts to do this.

1. Detect all module files changed and include them in the publish list
2. Figure out all modules that import/use those modules that have been impacted recursively
3. Publish

Also created a script that will automatically update all imports/uses to the latest version available in the ACR.

https://github.com/lukaferlez/Simpleverse.Bicep