Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding docs for working with hooks and azd environment #3949

Closed
wants to merge 3 commits into from
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 172 additions & 0 deletions cli/azd/docs/environment-with-hooks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Using azd environment from hooks

As an azd templates author, you might need to extend what your template can do by adding azd hooks. While hooks are very simple way to provide your own scripts and operations, there are a few details to keep in mind when your scripts will be interacting with azd's state, like the values from the environment (either just reading or setting values). In this article, you will learn some of the options you have for writing hooks, as well as interacting with the azd environment.

## Azd environment

In a few words, azd combines the `state` and the `configuration` for your application into one folder and calls it `environment`.

**The configuration** is a set of inputs to define how to create your application's resources or how services behave. For example, you can have a setting to define the `SKU` to be use for hosting one of your services, setting a `basic` SKU by default and expecting customers to set the right SKU by using that configuration.

**The state** are a set of values to describe properties for your application at a given point in time. For example, after calling `azd provision` all the infrastructure deployment's outputs are persisted in the environment. These values will not define or change the behavior of the application.

Azd allows you to create and switch between multiple environments. Each environment is created in a folder called `.azure`, next to the `azure.yaml` in your project. You can use `azd env --help` to discover how to create, list or switch between environments.

> !Note: Azd environment is different from CI/CD environments. **Do not** think about azd environment as GitHub environments.

When you run azd commands, azd uses the default selected environment. You can see the current default environment running `azd env list`. And you can override and set the environment to be use with:

- **AZURE_ENV_NAME**, You can set the name of the environment in your system's variables and azd will use it instead of the default selected environment.
- **-e**, You can use the `-e` flag when running azd commands to define the name of the environment to use. Azd will ignore your system's variables and the default selected environment when -e flag is set.

> If you want to manually change the default selected environment without running `azd env select <name>`, you can update the `.azure/config` file directly.

## Azd environment in hooks

When azd invokes a hook, the environment values are automatically injected and you can read them the same way you would read any other value from your terminal. Here is a very simple example that demonstrates this. Consider the next hook definition:

```yaml
name: demo-hooks
hooks:
preprovision:
shell: sh
run: ./script.sh
```

Now let's create `script.sh` like:

```bash
#!/bin/bash

echo $ENV_VAR_FROM_AZD

# just to let you see the output
sleep 5
```

And before you test it, make sure you set a value running:

```bash
azd env set ENV_VAR_FROM_AZD "hello azd"
# just to verify the value is in azd environment
azd env get-values
# you should see the `ENV_VAR_FROM_AZD="hello azd"` listed
```

Now you can run `azd provision` to trigger the hook. Or you can use `azd hooks run preprovision` so you will only execute the hook alone. You will see `hello azd` in the screen, as the `$ENV_VAR_FROM_AZD` is injected to the hooks execution. You can verify that if you manually run the script `./script.sh`, it won't display the message, as `ENV_VAR_FROM_AZD` won't have any value.

So, as you are creating a new hook and testing it, make sure to use `azd hooks run <hook name>` if you are reading azd environment. Otherwise, if you try manually running the script, you won't see the azd environment values.

In case you want to reproduce the same example in Windows powershell:

```yaml
name: demo-hooks
hooks:
preprovision:
shell: pwsh
run: ./script.ps1
```

Now let's create `script.ps1` like:

```pwsh
Write-Output $env:ENV_VAR_FROM_AZD

# just to let you see the output
sleep 5
```

### Self environment injection

In case you want to be able to *manually run* the `script` from a hook and without using azd, but you still want to read azd environment.

You might want to allow customers to manually run the scripts in case the hook failed when it was invoked by azd. For example, as a troubleshooting guide, you might be telling folks which script to run to correct missing settings or repair application components. For such cases, consider asking customers to use `azd hooks run <hook name>` as a way to make it simpler for customers and still delegating the environment injection to azd.

But, if you still want to allow the manual invocation, you will need to teach your script how to pull pull the environment values from azd by calling and parsing the output of `azd env get-values`. Below are 2 strategies you can implement on your script.

#### Use script scoped variables

Ideally, the azd environment should be read and used by the script without affecting any other process or the terminal session. See the next code:

```bash
#!/bin/bash

declare -A azdEnv
while IFS='=' read -r key value; do
value=$(echo "$value" | sed 's/^"//' | sed 's/"$//')
azdEnv[$key]=$value
done <<EOF
$(azd env get-values)
EOF

echo "${azdEnv["ENV_VAR_FROM_AZD"]}"

# just to let you see the output
sleep 5
```

By using `declare -A azdEnv`, you can allocate a set to write the keys and values from the `azd env get-values` output. Then you can access the values by referencing the environment keys with `"${azdEnv["ENV_VAR_FROM_AZD"]}"`. This strategy would allow you to test or run your script either manually or using `azd hooks run <hook name>`, and without affecting the terminal variables.

Here is the powershell equivalent.

```pwsh
$azdEnv = @{}
$azdEnvRaw = azd env get-values

foreach ($line in $azdEnvRaw -split "`n") {
if ($line -match '^(.*?)=(.*)$') {
$key = $matches[1]
$value = $matches[2] -replace '^"|"$'
$azdEnv[$key] = $value
}
}

Write-Output $azdEnv["ENV_VAR_FROM_AZD"]

# just to let you see the output
Start-Sleep -Seconds 5
```
Comment on lines +89 to +128
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we should be focusing on the automatic behavior of hooks scripts when run in the context of azd and use auto-injection as a standard and less about how to manually load azd env into the scripts since we already do that.

I'd also like to see examples of how we could make downstream calls from scripts... example - calling python script that will automatically make azd env vars available since they were injected into the script environment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100% Agree.
I'm trying to make it explain and recommend the use of azd hooks run .. for testing and how it is easier to let azd to inject the environment.

There are many templates out there, however, using a load-azd-env script. Some of them exists from before azd hooks run was available and because folks wanted to test the script whit azd env.

And yes, I am planning to add examples about the downstream calls for python :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any downstream calls in a sub process will automatically inherit any environment variables already set in the script including azd .env values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just feels like all this makes it more confusing when we should just be instructing users to call azd hooks run <hook> and it will do the right thing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just feels like all this makes it more confusing when we should just be instructing users to call azd hooks run and it will do the right thing.

I've been asked a few times, after telling folks to use azd hooks run ...: - but what if I want to manually run it? (usually for more complex scenarios, like a main script invoking other scripts, where only one of the sub-scripts requires azd-env.

I guess we can elevate the azd hooks run ... to the learn.microsoft official docs, while leaving this doc as a more advanced/complex guide

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running azd hooks run <hookname> will execute whatever script / code is referenced in the hook configuration in the azure.yaml. There isn't any current support to execute arbitrary scripts from with azd.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a separate topic (since I'm writing a different doc), but a common scenario or question I see is developers trying to use the outputs of the main.bicep in their language framework, such as Azure service endpoints. For example, an azd script creates a storage account and an OpenAI account and writes those endpoints out to the azd env file. Next, a node or python app wants to discover and use those endpoints to connect via code. Those languages wouldn't really rely on the azd env file - they have their own env files used by framework packages, or rely on the system/container environment variables. Is there a recommended script somewhere for copying azd env values to a separate .env file for a language framework, or one that writes them out to the system for "permanent" use by the app?

I usually see a hook script used to copy the azd env contents to another location at the end of the provisioning process, but the scenarios in this doc assume the end goal is working with azd environment variables within hook scripts - not outside of them in a language framework/app.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexwolfmsft , a common strategy for what you described is to use VSCode tasks settings and the Azure Developer CLI VSCode extension to define a task for launching your application locally after injecting the azd-env. This is the strategy used by the todo-templates for running services locally. As you can see, it depends on VSCode and azd-extension.

When no VSCode or when you don't want to define a task.json file, you can create your own script that start your service locally. The script can call azd env list -o json and parse the output to discovered the default azd environment to use. Then read and use the values to invoke the service. (Basically manually implementing what VSCode does with tasks.json). I think @pamelafox uses this strategy.

And the 100% manual alternative is where folks needs to cd into the service's folder they need to run and figure what env vars to set before manually launching/starting the service. For multiple services, folks would use a new console for each service.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a separate topic (since I'm writing a different doc), but a common scenario or question I see is developers trying to use the outputs of the main.bicep in their language framework, such as Azure service endpoints. For example, an azd script creates a storage account and an OpenAI account and writes those endpoints out to the azd env file. Next, a node or python app wants to discover and use those endpoints to connect via code. Those languages wouldn't really rely on the azd env file - they have their own env files used by framework packages, or rely on the system/container environment variables. Is there a recommended script somewhere for copying azd env values to a separate .env file for a language framework, or one that writes them out to the system for "permanent" use by the app?

I usually see a hook script used to copy the azd env contents to another location at the end of the provisioning process, but the scenarios in this doc assume the end goal is working with azd environment variables within hook scripts - not outside of them in a language framework/app.

@alexwolfmsft - Can you add your scenarios here: #4067

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please have a look at this proposal for azd script run #4131

Comment on lines +87 to +128
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


This strategy is fine and easy to implement when you have only one script, but if you have many scripts, you would need to copy and paste the same code in all the scripts, which might be hard to maintain. For such scenarios, take a look to the next strategy.

#### Use terminal scoped variables

When you have multiple scripts and you need all of some of them to access azd environment, you can make your script to set the azd environment in your terminal variables. You have to be careful with this strategy and ideally make sure to restore your terminal variables once all your scripts have run. This is because some terminals like powershell does not create one environment for executing the script, but update the environment of the terminal you are running in.

The main benefit of using your terminal variables is that you just need to set the variables one time, at your starting point, and then all scripts you run after will read the azd environment as direct variables from the terminal. See the next example:

```bash
#!/bin/bash

declare -A azdEnv
while IFS='=' read -r key value; do
value=$(echo "$value" | sed 's/^"//' | sed 's/"$//')
export "$key=$value"
done <<EOF
$(azd env get-values)
EOF

echo $ENV_VAR_FROM_AZD

# just to let you see the output
sleep 5
```

And the powershell version:

```pwsh
foreach ($line in (& azd env get-values)) {
if ($line -match "([^=]+)=(.*)") {
$key = $matches[1]
$value = $matches[2] -replace '^"|"$'
[Environment]::SetEnvironmentVariable($key, $value)
}
}

Write-Output $env:ENV_VAR_FROM_AZD

# just to let you see the output
Start-Sleep -Seconds 5
```

After running your script, make sure that azd environment is not leaked and persisted to your terminal variables, as it might affect future execution of azd commands because azd won't be switching to any other environment (unless you force it with the -e flag). This is because the `AZURE_ENV_NAME` would be persisted as terminal variable. So, depending on the terminal you are using (for sure on powershell), you would need to have a `restore variables` script at the end.
vhvb1989 marked this conversation as resolved.
Show resolved Hide resolved
Loading