Setting environment variables on a per-recipe basis without $-parameters

[neunenak]

I think it would be useful if there was a way to set environment variables on a per-recipe basis without using the $VARIABLE syntax in parameters. Maybe this could be an attribute:

[export: "ANDROID_SDK_HOME" "~/Android"]
my-recipe:
    echo $ANDROID_SDK_HOME

[casey]

That seems reasonable. What's the use-case? And, why not export ANDROID_SDK_HOME := "~/Android" in the outer scope?

'This would be slightly awkward as-is, since it would create a variable which is exported, but not available in just expressions, and whose name may be an arbitrary string. (As opposed to existing variables, which are always available in just expressions, and may or may not be exported.)

To make it fit in better with existing functionality, it might be best to instead add a way to add just variable bindings that are scoped to an individual recipe, which can optionally be exported.

[neunenak]

The use case I had in mind was being able to create individual recipes that have some environment variables in scope, without affecting the environment variables of other recipes (which is what export in the outer scope would do). The particular thing I wanted to do earlier that made me think of this issue was creating a general run-with-android-vars *args: recipe that would basically run arbitrary commands in an environment where ANDROID_SDK_HOME and a couple of other env vars were set to specific values. I don't want to clutter up my other recipes with those android-specific env vars, and what if I needed to make a different recipe that set the same env vars to some other value?

But yeah finding a way to scope just variable bindings to an individual recipe while keeping the existing distinction between export and non-export variables seems more general and useful.

I guess this could also be done with attributes, maybe something like:

[var: "SOME_VARIABLE", "foo"]
[export-var: "ANDROID_SDK_HOME", "~/Android"]
recipe:
    echo {{SOME_VARIABLE}}
    echo $ANDROID_SDK_HOME

[laniakea64]

The particular thing I wanted to do earlier that made me think of this issue was creating a general run-with-android-vars *args: recipe that would basically run arbitrary commands in an environment where ANDROID_SDK_HOME and a couple of other env vars were set to specific values. I don't want to clutter up my other recipes with those android-specific env vars, and what if I needed to make a different recipe that set the same env vars to some other value?

Can you make your recipe a shebang recipe? -

[no-cd, positional-arguments]
run-with-android-vars +args:
  #!/bin/bash
  set -euo pipefail
  export ANDROID_SDK_HOME={{quote(android_sdk_home)}}
  export OTHER_ENV_VAR='other env var value'
  exec "$@"

[neunenak]

The particular thing I wanted to do earlier that made me think of this issue was creating a general run-with-android-vars *args: recipe that would basically run arbitrary commands in an environment where ANDROID_SDK_HOME and a couple of other env vars were set to specific values. I don't want to clutter up my other recipes with those android-specific env vars, and what if I needed to make a different recipe that set the same env vars to some other value?

Can you make your recipe a shebang recipe? -

[no-cd, positional-arguments]
run-with-android-vars +args:
  #!/bin/bash
  set -euo pipefail
  export ANDROID_SDK_HOME={{quote(android_sdk_home)}}
  export OTHER_ENV_VAR='other env var value'
  exec "$@"

This is what I ended up doing, which does work, but it's a bit inelegant.

[tahv]

Hi,

I'm porting one of my Makefiles to Just and I have a use case that might be relevant.

I'm building a python app with pyapp, which can only be configured with environments variables. Here is what my Makefile target looks like:

MAKEFILE_DIR := $(realpath $(dir $(lastword $(MAKEFILE_LIST))))

.PHONY: build
build: export PYAPP_DISTRIBUTION_EMBED = 1
build: export PYAPP_FULL_ISOLATION = 1
build: export PYAPP_PROJECT_PATH = $(MAKEFILE_DIR)/dist/foo-$(shell $(python) -m hatch version)-py3-none-any.whl
build:
	python -m hatch build --clean --target wheel
	python -m hatch build --clean --target binary

• The PYAPP_ variables are scoped to the build job.
• The PYAPP_PROJECT_PATH variable executes a shell command to get the version of my app, and this command is only run if the build job is executed.
• I'm building the app twice, for Windows an Unix. Since exporting environment variables with Makefile is cross-platform, I can use this target in any shell.

The equivalent Justfile would look something like this:
I haven't tested it yet, but you get the idea.

[windows]
build:
  #!powershell.exe
  $env:PYAPP_DISTRIBUTION_EMBED = "1"
  $env:PYAPP_FULL_ISOLATION = "1"
  $env:PYAPP_PROJECT_PATH = "{{justfile_dir()}}/dist/foo-$(python -m hatch version)-py3-none-any.whl"
  uv run python -m hatch build --clean --target wheel
  uv run python -m hatch build --clean --target binary

[unix]
build:
  #!/usr/bin/env sh
  export PYAPP_DISTRIBUTION_EMBED='1'
  export PYAPP_FULL_ISOLATION='1'
  export PYAPP_PROJECT_PATH='{{justfile_dir()}}/dist/foo-$(python -m hatch version)-py3-none-any.whl'
  python -m hatch build --clean --target wheel
  python -m hatch build --clean --target binary

And based on @neunenak examples, it would look like this:
I have never used the shell function before, not sure if this is the correct syntax.

[export: "PYAPP_DISTRIBUTION_EMBED", "1"]
[export: "PYAPP_FULL_ISOLATION", "1"]
[export: "PYAPP_PROJECT_PATH", "{{justfile_dir()}}/dist/foo-{{shell('python -m hatch version')}}-py3-none-any.whl"]
build:
  python -m hatch build --clean --target wheel
  python -m hatch build --clean --target binary

I hope this help.

[W1M0R]

The particular thing I wanted to do earlier that made me think of this issue was creating a general run-with-android-vars *args: recipe that would basically run arbitrary commands in an environment where ANDROID_SDK_HOME and a couple of other env vars were set to specific values. I don't want to clutter up my other recipes with those android-specific env vars, and what if I needed to make a different recipe that set the same env vars to some other value?

Can you make your recipe a shebang recipe? -

[no-cd, positional-arguments]
run-with-android-vars +args:
  #!/bin/bash
  set -euo pipefail
  export ANDROID_SDK_HOME={{quote(android_sdk_home)}}
  export OTHER_ENV_VAR='other env var value'
  exec "$@"

@laniakea64 Although a shebang recipe works, it is not very cross-platform. If the recipe must work in Windows and in Linux, then that would require duplication of the recipe with minor modifications and overrides, or special steps to prepare the developer environments. Having attributes to deal with environment variables will greatly simplify cross-platform recipes.

[casey]

Since this is really about having a per-recipe scope for expressions, then I think we should think about reusing the existing expression syntax. I.e., FOO = "BAR", export FOO := "BAR", and unexport FOO, instead of coming up with new syntax.

I have no good ideas for syntax though 😂

[casey]

I'm almost tempted to close this in favor of #1023, since I think they're basically the same thing, a per-recipe scope for expressions. But there's good discussion here so whatever.

[laniakea64]

Since this is really about having a per-recipe scope for expressions, then I think we should think about reusing the existing expression syntax. I.e., FOO = "BAR", export FOO := "BAR", and unexport FOO, instead of coming up with new syntax.

I have no good ideas for syntax though 😂

#2409 (comment) mentioned the idea of scoping to recipe groups instead of individual recipes. If that would do the job, what about setting the [group: "group-name"] attribute on variables / export statements / unexport statements?

[group: "android"]
export ANDROID_SDK_HOME := home_dir() / "Android"

# The above ANDROID_SDK_HOME export variable is available and exported for this recipe...
[group: "android"]
run-with-android-vars +args:
  env

# ... but not this one
run-without-android-vars +args:
  env

Edit: Oops, that technically wouldn't be backwards-compatible, since it is not an error in current just (the [group] attribute on the variable is accepted but seems ignored). If that would be a problem, this could be a new attribute [scope: "scope-name"] - or even [scope("scope name 1", "scope name 2", ...)] to put one scoped variable in multiple scopes or combine multiple scopes for one recipe

[casey]

That's an interesting option. I think it's a downside that you have to add a group to a recipe if you want scoped variables, and you have assignments which potentially share a scope, but aren't syntactically grouped.

That being said, I think invalid attributes being accepted on assignments is an inadvertent bug which should be fixed, so this is potentially backwards compatible.

[casey]

That being said, I think invalid attributes being accepted on assignments is an inadvertent bug which should be fixed, so this is potentially backwards compatible.

Fixed in #2412.

[casey]

A wacky option would be to allow defining inline modules, place the recipe you want to have its own scope in the module, and then use normal assignments within that module:

my-recipe: foo::my-recipe

# or maybe:
use foo::my-recipe
use foo::*

# hypothetical inline module syntax
foo::
  export ANDROID_SDK_HOME := "~/Android"
  
  my-recipe:
    echo $ANDROID_SDK_HOME

This allows multiple recipes to share a scope, and re-uses existing functionality. However, given the current limitations of modules, a workaround may be desirable.

[neunenak]

A wacky option would be to allow defining inline modules, place the recipe you want to have its own scope in the module, and then use normal assignments within that module:

my-recipe: foo::my-recipe

# or maybe:
use foo::my-recipe
use foo::*

# hypothetical inline module syntax
foo::
  export ANDROID_SDK_HOME := "~/Android"
  
  my-recipe:
    echo $ANDROID_SDK_HOME

This allows multiple recipes to share a scope, and re-uses existing functionality. However, given the current limitations of modules, a workaround may be desirable.

I kind of like this, since it allows a group of recipes to share a common set of variables, which I could see being useful. Actually, maybe I could solve my problem today by simply using modules (which I don't personally normally use in just), although I'd have to resort to having multiple files which is a bit annoying.

[W1M0R]

Another option could be to have an env_file attribute:

[env_file("./myvars.env")]
my-recipe:
    echo $ANDROID_SDK_HOME

[casey]

I think I'm inclined to implement this with inline modules, since it's more general and powerful, although it might take awhile for someone to get around to it. See #2442 for some discussion.

[alexthecarp]

I find i also would need this functionality. The ticket is closed as completed, but #2442 is still open so that's not how it's been completed. May I ask what the resolution was ?

For context, I have some build-time vars and some run-time vars. I use the export feature to pull the run-time ones from Vault when running locally.
In CI, i only need the build ones, so the top of the file export section doesn't work for run-time ones because I don't want CI to pull them.
This feature would be great to only pull the run-time ones when i'm actually trying to execute recipes that also run the code.

[casey]

#2442 is big, so it may take a while.

[alexthecarp]

Thanks for the reply. No problem, I just wasn't sure where to go look for updates. I'll watch #2442 <#2442> .

…

On Mon, Nov 18, 2024 at 1:05 PM Casey Rodarmor ***@***.***> wrote: #2442 <#2442> is big, so it may take a while. — Reply to this email directly, view it on GitHub <#2408 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHAAOJ22OAD4V4QJM2UNMCD2BI3BHAVCNFSM6AAAAABPI4AT7GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBTHA4DCOJSG4> . You are receiving this because you commented.Message ID: ***@***.***>