Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: Add DAG_DETAILS_JSON for access to DAG metadata within tasks #710

Open
yohamta opened this issue Nov 15, 2024 · 0 comments
Open
Assignees

Comments

@yohamta
Copy link
Collaborator

yohamta commented Nov 15, 2024

Currently, accessing DAG details (like schedule, last run time, etc.) within tasks requires workarounds or potentially undocumented environment variables. This makes it difficult for tasks to react dynamically to DAG-specific information.

We would like to add a new environment variable, DAG_DETAILS_JSON, which will contain a JSON representation of all relevant DAG metadata. This will provide a consistent and comprehensive way for tasks to access this information, especially useful for executors like jq that can easily parse JSON.

Example Usage (with jq):

steps:
  - name: get schedule
    executor: jq
    command: '.schedule'
    script: "echo $DAG_DETAILS_JSON"
    output: DAG_SCHEDULE

Related discussions: #708

@yohamta yohamta changed the title Add DAG_DETAILS_JSON for access to DAG metadata within tasks executor: Add DAG_DETAILS_JSON for access to DAG metadata within tasks Dec 29, 2024
@yohamta yohamta self-assigned this Dec 30, 2024
@yohamta yohamta removed the soonish label Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant