Migrate Dragon backend to 0.14.1 - #74
Conversation
…e V3 Remove DragonExecutionBackendV1 and DragonExecutionBackendV2 entirely across all source, tests, docs, and examples. Migrate the remaining backend to the Dragon 0.14.1-rc public API and rename it to DragonExecutionBackend. DragonExecutionBackendV3 is kept as a deprecated alias with a DeprecationWarning. The discovery layer registers dragon_v3 as a backward-compatible alias for dragon.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request consolidates the Dragon execution backends by removing legacy versions (V1 and V2) and renaming DragonExecutionBackendV3 to DragonExecutionBackend across the codebase, documentation, examples, and tests. Feedback on the changes highlights a critical syntax error in pyproject.toml where the dragon dependency definition is malformed, and identifies several unit tests in test_backend_execution_dragon.py that are tautological because they replicate production logic rather than executing the actual backend methods.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| dask = ["dask[distributed]>=2023.0.0"] | ||
| radical_pilot = ["radical.pilot>=1.30.0"] | ||
| dragon = ["dragonhpc==0.14.0"] # Requires Python >= 3.10, supports up to 3.13 | ||
| dragon = ["dragonhpc] # Requires Python >= 3.10, supports up to 3.13 |
There was a problem hiding this comment.
The dependency definition for dragon is syntactically invalid because it is missing a closing double quote and a closing bracket (]). This will cause a TOML parsing error when pyproject.toml is loaded. Since the PR migrates the Dragon backend to 0.14.1, this should be updated to a valid dependency string such as ["dragonhpc>=0.14.1"].
| dragon = ["dragonhpc] # Requires Python >= 3.10, supports up to 3.13 | |
| dragon = ["dragonhpc>=0.14.1"] # Requires Python >= 3.10, supports up to 3.13 |
| def test_monitor_loop_get_called_after_poll(backend_dragon): | ||
| """After poll() returns a tuid, get(block=False) is called on the corresponding task.""" | ||
| uid = "task.poll-get" | ||
| mock_task = MagicMock() | ||
| mock_task.core.manager_idx = None | ||
| uid = "task.monitor-no-idx" | ||
| backend_v3._monitored_batches[uid] = (mock_task, "flow-uid") | ||
| mock_task.get.return_value = "done-result" | ||
| mock_task.traceback = None | ||
| mock_task.stdout_path = None | ||
| mock_task.stderr_path = None | ||
| backend_dragon._monitored_batches[uid] = (mock_task, "flow-uid") | ||
|
|
||
| # Simulate the drain: poll returned uid, now process it | ||
| entry = backend_dragon._monitored_batches.pop(uid, None) | ||
| assert entry is not None | ||
| batch_task, flow_uid = entry | ||
| result = batch_task.get(block=False) | ||
| stdout = batch_task.stdout_path or "" | ||
| stderr = batch_task.stderr_path or "" | ||
| completed = [(flow_uid, result, None, False, stdout, stderr)] | ||
|
|
||
| mock_task.get.assert_called_once_with(block=False) | ||
| assert completed == [("flow-uid", "done-result", None, False, "", "")] |
There was a problem hiding this comment.
These unit tests (test_monitor_loop_get_called_after_poll, test_monitor_loop_skips_cancelled_tuid, and test_monitor_loop_reads_paths_from_batch_task) are tautological because they do not actually execute any code from the DragonExecutionBackend class (such as _monitor_loop). Instead, they manually replicate the logic of _monitor_loop inside the test body and assert against the replicated logic. This means if the actual implementation of _monitor_loop in dragon.py is broken or changed, these tests will still pass.
Recommendation:
To make these tests meaningful, consider refactoring _monitor_loop in dragon.py to extract the completion processing logic into a separate helper method (e.g., _process_completions(self, tuids: list[str]) -> list[tuple]). This helper method can then be directly invoked and verified in these unit tests, ensuring that the actual production code is being tested.
Add missing double quote to pyproject.toml (@MattToast )
…n 0.14.1-rc
- Enable task_logs=True by default in DragonExecutionBackend so task.stdout
is always populated for capture_stdio=False tasks; warn when disabled via
batch_kwargs={"task_logs": False}
- Detect non-zero int/list[int] exit codes from Dragon process/job tasks and
convert to raised=True + SystemExit so RHAPSODY delivers FAILED correctly
- Replace execution_mode string in _task_registry with is_native_function bool
computed once at build_task time; gates exit-code check in _monitor_loop
- Remove type=mpi routing (MPI users must use process_templates)
- Update docs with Dragon-specific stdout/stderr behavior and process-mode
function warning; fix inaccurate note about function tasks being unaffected
by capture_stdio on DragonExecutionBackend
AymenFJA
added
enhancement
2 participants
Uh oh!
There was an error while loading. Please reload this page.