Skip to content

Rename build_all_dags to build_all_airflow_dags for safe-mode discovery - #65

Merged
jeremybeard merged 3 commits into
mainfrom
rename-build-all-airflow-dags
Jun 22, 2026
Merged

Rename build_all_dags to build_all_airflow_dags for safe-mode discovery#65
jeremybeard merged 3 commits into
mainfrom
rename-build-all-airflow-dags

Conversation

@jeremybeard

Copy link
Copy Markdown
Collaborator

What

Renames the DAG-loading entry point from build_all_dags to build_all_airflow_dags so that a minimal one-line loader is actually discovered by Airflow. build_all_dags and build_all remain as deprecated aliases that emit DeprecationWarning and forward.

Why

Airflow's DAG file processor runs in safe mode by default. Its discovery heuristic (airflow.utils.file.might_contain_dag) only treats a file as a potential DAG file when its contents contain both the airflow and dag substrings:

content = content.lower()
if b"airflow" not in content:
    return False
return any(s in content for s in (b"dag", b"asset"))

The previous entry point build_all_dags carries dag but not airflow. So the canonical loader we ship and document:

from blueprint import build_all_dags
build_all_dags()

contains no airflow substring and is silently skipped by Airflow — no error, just missing DAGs. It only worked when something unrelated (e.g. from airflow import DAG for a type hint) happened to drag the airflow substring into the file. That is exactly the trap users hit.

build_all_airflow_dags puts both required substrings in the import line itself, so the bare-minimum loader works with nothing else in the file:

from blueprint import build_all_airflow_dags
build_all_airflow_dags()

Independent integration test for the bare-minimum loader

tests/integration/test_safe_mode_discovery.py is new and self-contained. It writes the bare-minimum loader (import + call, nothing else — no from airflow import DAG, no docstring) and asserts Airflow's real safe-mode scanner picks it up:

  • test_bare_minimum_loader_is_discovered_by_safe_mode — the new build_all_airflow_dags loader passes might_contain_dag(..., safe_mode=True).
  • test_legacy_loader_is_skipped_by_safe_mode — the pre-rename build_all_dags loader is skipped, documenting the regression this fixes.

It calls Airflow's actual discovery function rather than a re-implementation, so it cannot drift from real behaviour, and it needs no running Airflow instance (no Astro standalone), so it runs fast and in isolation from the API-driven integration tests.

Changes

  • blueprint/builder.py: real function renamed to build_all_airflow_dags; build_all_dags + build_all are deprecating aliases.
  • blueprint/__init__.py: export build_all_airflow_dags (aliases still exported).
  • blueprint/cli.py: blueprint new scaffold output uses the new name.
  • examples/, README.md: loaders and docs use build_all_airflow_dags. The simple example is the bare-minimum loader.
  • tests/test_builder.py: functional tests target the real function; added a deprecation test for build_all_dags.
  • tests/integration/project/dags/loader.py: uses the new name.

Verification

  • 266 unit tests pass; ruff and ty clean.
  • New safe-mode tests pass (2/2), including the negative case proving the old name was skipped.

Notes

  • This is the second rename of this symbol (build_allbuild_all_dagsbuild_all_airflow_dags). The first rename only added the dag substring and never fixed discovery, because its premise — that Airflow needs either dag or airflow — was wrong; it needs both. Both old names are kept as forwarding aliases, so existing loaders keep working with a deprecation warning.

Airflow's DAG file processor runs in safe mode by default and only treats
a file as a potential DAG file when its contents contain BOTH the 'airflow'
and 'dag' substrings (airflow.utils.file.might_contain_dag). The previous
entry point build_all_dags carries 'dag' but not 'airflow', so the minimal
one-line loader

    from blueprint import build_all_dags
    build_all_dags()

is silently skipped by Airflow -- no error, just missing DAGs. Loaders only
worked when something else (e.g. 'from airflow import DAG') happened to drag
the 'airflow' substring into the file.

Rename the entry point to build_all_airflow_dags so the import line itself
carries both required substrings. build_all_dags and build_all remain as
deprecated aliases that emit DeprecationWarning and forward.

Adds tests/integration/test_safe_mode_discovery.py: an independent test that
writes the bare-minimum loader (import + call, nothing else) and asserts
Airflow's real safe-mode scanner discovers it -- and that the pre-rename
loader is skipped, documenting the regression this fixes.
Address review: the safe-mode test no longer calls Airflow's might_contain_dag
helper directly. Instead the project ships a real bare-minimum loader at
dags/safe_mode_minimal/loader.py (an import + call, deliberately no
'from airflow import DAG') that builds a probe DAG, and the test asserts the
running Airflow instance discovered and parsed it via the REST API, with no
import errors.

The loader is isolated from the project's main loader by a distinct
'*.safe.yaml' pattern so the two never build the same dag_id. A small
on_dag_built tag callback (no airflow import) satisfies the project's
'every DAG has tags' integrity convention.
Match the cleanup pattern used by _get_caller_file(): wrap the
inspect.currentframe() capture in build_all_dags and build_all in
try/finally with 'del frame' to avoid lingering frame references.
@astronomer astronomer deleted a comment from github-actions Bot Jun 22, 2026
@astronomer astronomer deleted a comment from github-actions Bot Jun 22, 2026
@astronomer astronomer deleted a comment from github-actions Bot Jun 22, 2026
@jeremybeard
jeremybeard marked this pull request as ready for review June 22, 2026 18:05
@jeremybeard
jeremybeard merged commit 2a1493d into main Jun 22, 2026
14 checks passed
@jeremybeard
jeremybeard deleted the rename-build-all-airflow-dags branch June 22, 2026 18:05
@jlaneve jlaneve mentioned this pull request Jul 20, 2026
jlaneve added a commit that referenced this pull request Jul 20, 2026
New since v0.3.0: .airflowignore support for DAG YAML discovery and
blueprint lint (#67), build_all_airflow_dags rename for safe-mode
discovery (#65), source paths in blueprint list (#64), and -h/-v
shorthands (#63).


Claude-Session: https://claude.ai/code/session_01J44AEQ8KTvUWxf6GnVRW66

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant