Skip to content

feat: orchestrate between the numeric and prompt optimizations - #1860

Open
gabwow wants to merge 1 commit into
AALGO-599-implement-ga-algo/agabowfrom
AALGO-559-orchestrate-optimization/agabow
Open

feat: orchestrate between the numeric and prompt optimizations#1860
gabwow wants to merge 1 commit into
AALGO-599-implement-ga-algo/agabowfrom
AALGO-559-orchestrate-optimization/agabow

Conversation

@gabwow

@gabwow gabwow commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Related Issue

Changes

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with documentation updates
  • Documentation only
  • Contributor tooling or automation
  • CI, build, or test infrastructure

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Documentation updated for user-visible behavior
  • Documentation not applicable — justification:

Verification

  • Pull request title follows the repository's Conventional Commit format
  • Every commit includes an appropriate Signed-off-by: trailer
  • uv run pre-commit run -a passes, or any blocked checks are identified below
  • Targeted tests pass, or tests are marked not applicable above
  • No secrets, API keys, or credentials are included

Targeted validation:

Summary by CodeRabbit

  • New Features

    • Added sequential numeric and prompt optimization phases with independent enablement and handoff of results between phases.
    • Optimization runs now preserve intermediate and final artifacts, experiment details, trial offsets, and phase outcomes.
    • Failed phases produce structured results and persisted failure artifacts instead of stopping without context.
    • Failure reporting now includes the number of trials executed.
  • Bug Fixes

    • Improved handling of malformed optimizer configurations with clear validation errors.
    • Prevented invalid or missing enabled values from being silently treated as disabled.

@gabwow
gabwow requested review from a team as code owners September 8, 2026 01:27
@github-actions github-actions Bot added the feat label Sep 8, 2026
@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5fb138b4-d961-4ae2-9dcb-f2a1ad311081

📥 Commits

Reviewing files that changed from the base of the PR and between f7a5c20 and 25dd2be.

📒 Files selected for processing (11)
  • plugins/nemo-optimization/pyproject.toml
  • plugins/nemo-optimization/src/nemo_optimization/backends/optuna/backend.py
  • plugins/nemo-optimization/src/nemo_optimization/backends/optuna/study_driver.py
  • plugins/nemo-optimization/src/nemo_optimization/bundle.py
  • plugins/nemo-optimization/src/nemo_optimization/jobs/optimize.py
  • plugins/nemo-optimization/src/nemo_optimization/router.py
  • plugins/nemo-optimization/src/nemo_optimization/search_space.py
  • plugins/nemo-optimization/tests/test_bundle.py
  • plugins/nemo-optimization/tests/test_optimize_job.py
  • plugins/nemo-optimization/tests/test_router.py
  • plugins/nemo-optimization/tests/test_study_driver.py

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.


📝 Walkthrough

Walkthrough

The optimization router now supports sequential numeric and prompt phases. It validates phase configuration, carries payloads and trial offsets between phases, records failures and skipped phases, and persists intermediate and combined artifacts.

Changes

Optimization phase validation

Layer / File(s) Summary
Phase configuration validation
plugins/nemo-optimization/src/nemo_optimization/bundle.py, plugins/nemo-optimization/src/nemo_optimization/search_space.py, plugins/nemo-optimization/src/nemo_optimization/backends/optuna/study_driver.py, plugins/nemo-optimization/src/nemo_optimization/router.py, plugins/nemo-optimization/tests/test_bundle.py, plugins/nemo-optimization/tests/test_router.py
Numeric and prompt sections must be mappings with explicit boolean enabled values. Invalid sections now produce validation errors. Search-space errors become structured study errors.

Trial and failure metadata

Layer / File(s) Summary
Trial and failure metadata
plugins/nemo-optimization/src/nemo_optimization/backends/optuna/study_driver.py, plugins/nemo-optimization/src/nemo_optimization/backends/optuna/backend.py, plugins/nemo-optimization/tests/test_study_driver.py
Optuna failures preserve executed trial counts. Failed phase results and failure artifacts report those counts.

Sequential phase orchestration

Layer / File(s) Summary
Sequential phase orchestration
plugins/nemo-optimization/src/nemo_optimization/router.py, plugins/nemo-optimization/tests/test_router.py, plugins/nemo-optimization/pyproject.toml
The router runs enabled numeric and prompt phases in sequence, passes optimized payloads and trial offsets, records skipped or failed phases, and writes phase and combined artifacts.

Job outputs and integration coverage

Layer / File(s) Summary
Job outputs and integration coverage
plugins/nemo-optimization/src/nemo_optimization/jobs/optimize.py, plugins/nemo-optimization/tests/test_optimize_job.py
Job descriptions and error text now describe workflow optimization. Tests cover intermediate artifacts after failure and final artifacts after successful phases.

Sequence Diagram(s)

sequenceDiagram
  participant dispatch_payload
  participant _run_phases
  participant NumericBackend
  participant PromptBackend
  participant ArtifactWriter
  dispatch_payload->>_run_phases: phase configuration and initial payload
  _run_phases->>NumericBackend: numeric phase request with trial offset
  NumericBackend-->>_run_phases: numeric result and optimized payload
  _run_phases->>PromptBackend: prompt phase request with handed-off payload
  PromptBackend-->>_run_phases: prompt result and optimized payload
  _run_phases->>ArtifactWriter: phase details and combined result
  ArtifactWriter-->>dispatch_payload: persisted artifacts and final result
Loading

Merge Risk: ⚪ Minimal · up to 25dd2

No actionable merge-blocking risk remains in the multi-phase orchestration changes.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 72 functions across 10 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: orchestration between numeric and prompt optimization phases.
Full details: Docstring Coverage

Explanation

Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 72 functions across 10 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch AALGO-559-orchestrate-optimization/agabow

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 39058/49729 78.5% 62.6%
Integration Tests 23662/46966 50.4% 23.1%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant