Write every model as a recipe: one body per class, AOT-compilable, DC and DC-multi-period included (breaking: form is a type, 0.4.0) by sshin23 · Pull Request #57 · madsuite-org/ExaModelsPower.jl

sshin23 · 2026-08-13T12:15:01Z

What this does

Every model in the package — static AC (polar, rect), static DC, and multi-period (polar, rect, DC) — is now written as a recipe: the structure in an ExaCore with the case data standing in as an ExaModels.ArgSource placeholder, closed by an argument function (opf_args / mpopf_args) that turns a case file into the data. *_model is the two composed, and *_core builds the same body eagerly for ExaModelsC to compile as a fixed model.

The point is ahead-of-time compilation. With madsuite-org/ExaModels.jl#308, all six models compile into one shared library:

compile_library("libemp",
    :acp   => (opf_recipe(form = :polar)[1], opf_args, "pglib_opf_case14_ieee.m"),
    :acr   => (opf_recipe(form = :rect)[1],  opf_args, "pglib_opf_case14_ieee.m"),
    :dcp   => (opf_recipe(form = :dc)[1],    opf_args, "pglib_opf_case14_ieee.m"),
    :mpacp => (mpopf_recipe(N = 5, Nbus = 14)[1],               mpopf_args_default, "pglib_opf_case14_ieee.m"),
    :mpacr => (mpopf_recipe(N = 5, Nbus = 14, form = :rect)[1], mpopf_args_default, "pglib_opf_case14_ieee.m"),
    :mpdcp => (mpopf_recipe(N = 5, Nbus = 14, form = :dc)[1],   mpopf_args_default, "pglib_opf_case14_ieee.m"))

compiles in ~200 s with --trim=safe, and each model instantiates any matpower case from a path at run time — only the string crosses the C boundary; the parse happens inside the library. A library built against case14 instantiates case9241-pegase (85,568 variables) in under a second, exact against the in-Julia model.

One body per model class

The three static formulations shared their spine already; they are now one dispatched body (build_opf over Polar/Rect/DC), and the same for multi-period, which keeps its own spine (variables and ramp before voltage). DC is a formulation like any other: opf_model(f; form = :dc) is legal, dcopf_model(f) is the spelling that says it in the name. New capability that fell out: DC multi-period, including storage modelled as an active-power source (energy state, charge/discharge, ramp unchanged; the converter model — qint, I2, ohms — has no DC counterpart and is absent rather than approximated).

src/dcopf.jl is gone (folded into opf.jl); scopf.jl is renamed goc3.jl; GOC3 itself is untouched.

Nothing about the models changes

Verified throughout, not assumed:

Static AC, both formulations: identical to the previous constructors on dimensions, nnzj/nnzh, x0, all four bound vectors, objective, gradient, constraints, and Jacobian and Hessian including COO order (case14; the comparison is armed by five mutants, each moving exactly the quantities it should).
Static DC and all eight multi-period reference models (case3/case5 × polar/rect × storage/none): identical on the same eighteen quantities.
Compiled libraries: every model exact against its in-Julia counterpart through the raw C symbols, at case14 and (static) at case118, which no library was compiled against.

One intentional behaviour fix (measured, not silent): DC's pf bounds were taken from the arc-length rate_a and landed correctly only because the first nbranch arcs are the from-arcs in branch order; they now come from a per-branch branch_rate_a. Values identical on case14/case118/case9241.

Inference fixes that AOT forced and the package keeps

parse_ac_power_data inferred as Any (a Core.Box from a reused binding, T passed as a value, a Union-typed keyword, mismatched ternary branches); parse_mp_power_data likewise. Both now infer concretely — --trim=safe requires it, but it is a win on every call. The argument tuples stay under the 31-field NamedTuple inference ceiling by returning exactly the fields the bodies read (multi-period groups its N-expanded bound matrices under a nested field).

Tests

test/recipe_tests.jl: recipe-vs-eager equivalence across both static formulations (dims, bounds, x0, and all five callbacks at a point away from x0), a regression test for solution handles (the recipe's handles carry symbolic offsets and must be instantiated before solution can index with them — this shipped once and the suite caught it, twice), and an opt-in AOT compile behind EMP_TEST_AOT=1. NLPModels joins the test dependencies.

Suite: CPU slice 604/604; CUDA slice run on 2× GV100 (requires the instantiate fixes in madsuite-org/ExaModels.jl#308 — the recipe path is new on GPU, and the fix for the deferred-collect host pull-back is part of that PR).

Depends on

ExaModelsC: emit several models into one library ExaModels.jl#308 (argument-function surface, multi-model compile_library, and the instantiate trimmability/GPU fixes) — for the AOT path and the CUDA leg. The in-Julia models and this package's own suite do not otherwise depend on it.

🤖 Generated with Claude Code

…core The model gains `ac_opf_recipe` — its structure, with the case data left open — and `ac_opf_args`, the values that close it. `ac_opf_model` becomes the two composed, and `ac_opf_core` is the same model built eagerly, which is the form `ExaModelsC.compile_library` compiles. One body per formulation serves all three, so the compiled and in-Julia models cannot drift. Nothing about the models changes. Against the previous constructors, both `:polar` and `:rect` agree on dimensions, nnzj/nnzh, starting point, all four bound vectors, objective, gradient, constraints, and the Jacobian and Hessian down to the order of their COO triplets, on pglib_opf_case14_ieee. Three things could not be written against a placeholder and are now supplied as data rather than dropped: the starting points, the rectangular form's squared voltage bounds (broadcasting a placeholder is refused by design), and the thermal limits' -Inf lower bounds, all of which were built against a length not known until instantiation. `start` accepts a scalar, a vector, a `Base.Generator`, or a function of the parsed data, so a caller can warm-start from the case's own operating point with `start = (vm = d -> d.vm0,)`. A core stores the collection each generator iterated over, so iterating the parser's tables put `ExaPowerIO.BusData` and friends inside it. That is invisible in Julia and fatal ahead of time: the app ExaModelsC generates deserializes the core, and Serialization resolves a type's module only among loaded modules, so the app failed to precompile with KeyError: key Base.PkgId(UUID("14903efe-..."), "ExaPowerIO") not found Rows now cross as plain NamedTuples of the same field names. Every field is an Int, a T or an NTuple{3,T}, so they stay isbits and device-transferable, and `b.f_bus` and `g.c[1]` read exactly as before. Compiled and checked against the in-Julia model on ExaModels main 63f6f993: case14 (118 vars, 169 cons) and case118 (1088/1539), each exact on all fourteen quantities including the Hessian at obj_weight 0.7, evaluated at a point away from the starting point and re-checked at a second point.

Building an AC OPF recipe into a shared library needs every call on the instantiation path to resolve statically. Five things here did not, and each was invisible in the source. parse_ac_power_data assigned `data` twice while the ref_buses comprehension captured it, so Julia boxed it as a Core.Box whose contents are Any — enough on its own to make the whole function infer as Any. The parse result is now a separate binding. T was a plain positional argument, so it arrived typed Type rather than Type{Float64}; parse_matpower could not specialize and returned an unparameterized PowerData, after which every field read off it widened. It is now a ::Type{T} parameter through parse_ac_power_data and ac_opf_args. convert_data built its result from a generator, which yields concrete names and abstract contents; it maps instead. library = isfile(...) ? nothing : :pglib gave the keyword a Union type; the call is branched instead. The storage fields were ternaries whose branches returned different types; an empty Vector{StorageData} iterates to an empty Vector{T} on its own, so the guards are gone. ac_opf_args returned all 35 parsed fields. map over a NamedTuple stays inferable to 31 fields and gives up at 32 (measured), so the result had concrete names and no value types. It now returns exactly the 25 fields the model bodies read; the rest were inputs to the starting points, which are resolved beforehand and travel as the *_start arrays. TMPDIR was an untyped global assigned in __init__, leaving mkpath(TMPDIR::Any) unresolved — __init__ is part of a compiled image. It is a const Ref{String}. Models are unchanged: 36/36 identical to the previous constructors on both formulations, at pglib_opf_case14_ieee.

Rows were converted to plain NamedTuples so that no ExaPowerIO type would land inside the core — a core naming a package the generated app does not import cannot be deserialized there. ExaModels #305 has since merged, and the generated app now receives a path source for every developed package, so the core may name ExaPowerIO freely. Measured by reverting each change alone against the same compile: without this one the build is still clean (0 verifier errors, compiled), where without any of the other four it fails — 204, 84, 6 and 1 errors respectively. So this was the one change that had stopped earning its place. Removes a per-call copy of every table as well.

dcopf's pf is a branch-length block that took its bounds from rate_a, which parse_ac_power_data builds per ARC — twice as long. The values landed correctly only because the first nbranch arcs are the from-arcs in branch order; measured identical on case14, case118 and case9241_pegase, 0 differences. ac_opf_args now carries branch_rate_a and dcopf uses it, so it is right by construction rather than by that ordering. The DC models are unchanged: identical before and after across both cases on all eighteen quantities including COO order. Armed by scaling branch_rate_a by 1.01, which moves lvar and uvar in both cases and nothing else — so the comparison can see a bounds change. dcopf_model now builds its data through ac_opf_args, which is where the fields the recipe cannot compute from a placeholder are assembled. Adds test/recipe_tests.jl: ac_opf_model (recipe instantiated at args) against ac_opf_core (the same body built eagerly) on dims, bounds, x0 and all five callbacks, in BOTH formulations; a regression test for the solution handles; and an opt-in ahead-of-time compile behind EMP_TEST_AOT. NLPModels joins the test dependencies.

The three builders followed the same spine already: variables -> objective -> ref angle -> flows -> angle diff -> balance -> extras so there is now one body and a method per formulation for the steps that differ. Polar and rect shared 58% of their lines outright; DC under a third, but folding it in cost one extra method on five functions rather than a body full of no-ops, which is what the line counts said it would be: 154 lines against 213. Nothing about the models changes. Both AC formulations are identical to the pre-recipe constructors on dimensions, bounds, starting point, objective, gradient, constraints and the Jacobian and Hessian down to COO order, and DC is identical to its own previous builder on the same eighteen quantities, at case14 and case118. That the constraint ORDER already agreed across all three is what let the check stay exact rather than merely equivalent. DC gains a pf_start, which is zero — the default it already took — so it is settable now without changing anything. The one place this reads worse than the duplicated form is ac_flow, where a Val selects among four branch-flow expressions per formulation; eight one-line methods against eight inline ones, and it is what keeps add_flow_constraints! shared.

Multi-period keeps its own spine — generation, flows, objective, thermal limits and ramp all precede the voltage block, where the static body puts voltage first — but within it polar and rect were the same if/else the static builders had, and are now dispatched the same way on Polar/Rect. All eight multi-period models are unchanged: case3 and case5, polar and rect, with and without storage, identical on dimensions, bounds, starting point, objective, gradient, constraints and the Jacobian and Hessian including COO order. Getting there needed one real fix. The rectangular form inserts its voltage-magnitude rows BETWEEN the balance rows and the appends into them, and appending a term adds Jacobian entries — so doing the appends first reordered the COO triplets while leaving every other quantity identical. The balance rows and the appends are separate steps now, which is the mirror of the static body, where the appends come first. DC multi-period is new. The network half is the static DC model per period, the ramp limit is the AC one unchanged since it is on pg, and storage is modelled as an active power source: pst into the bus, E as the state, pstd/pstc in and out, with the resistive loss term dropped because it is a function of the current magnitude the DC linearization does not carry, and the pst^2 + qst^2 transfer limit becoming a bound on pst. The converter model — qint, I2 and the ohms relation — has no DC counterpart and is absent rather than approximated. Also replaces four copies of with opf_form, which is what had kept :dc unreachable.

The bodies merged already; the entry points had not. opf_recipe, opf_args, opf_core and opf_model are now the shared family, with the formulation as an argument, and ac_opf_* and dcopf_* are one-line spellings of them. ac_opf_args becomes opf_args, since it assembles the arguments for DC as readily as for AC. Callers see no change: ac_opf_model(f) is polar, dcopf_model(f) is DC, and ac_opf_model(f; form = :dc) is now legal and means what it says. src/dcopf.jl is 59 lines, down from 145 before the merge, and holds only the methods that differ from the shared body plus three aliases. Verified after the rename: static AC identical to the pre-recipe constructors 36/36 on both formulations, static DC identical on both cases, and all eight multi-period models identical.

dcopf.jl held only the DC methods of the shared body and three aliases, so it folds into opf.jl — it is static OPF. scopf.jl becomes goc3.jl, which is what it builds. No code changes in either move. parser.jl matpower -> the tables every model is built from constraint.jl the algebraic expressions, shared by all three goc3_parser.jl GOC3-specific parsing sc_parser.jl " opf.jl static: polar, rect, DC mpopf.jl multi-period: polar, rect, DC goc3.jl security-constrained, GOC3 formulation Verified across the move: static AC 36/36, static DC identical on both cases, all eight multi-period models identical, DC multi-period with storage still builds.

mpopf_recipe / mpopf_args / mpopf_core complete the recipe split for the multi-period class, so all six ExaModelsPower models — static and multi-period, polar, rect and DC — compile through ExaModelsC. N, the curve, the bus count and whether the case has storage are BUILD-time facts: the body slices genarray[:, 2:N], repeat has no symbolic form, and storage changes how many variable blocks exist. A compiled multi-period library is therefore per-(N, curve, Nbus, storage-shape), and the case file is the one value crossing the C boundary. mpopf_args_default is the one-argument, package-owned spelling a compiled library can call — a closure over a curve cannot be reached from the generated app. The N-expanded bound matrices are grouped under a nested rep field: there are 26 of them, and map over a NamedTuple stops inferring elementwise past 31 fields. convert_data recurses into nested NamedTuples for the same reason. Three inference faults fixed on the way, the same classes the static parser had: T passed as a value rather than a ::Type{T} parameter; the empty-storage ternary whose branches had different types (a vector of one NamedTuple shape against a matrix of another); and a head/tail slice pair where one branch produced a Matrix and the other a Vector. The parse and the whole argument path now infer concretely. All eight multi-period reference models are unchanged: case3/case5, polar and rect, with and without storage, identical on all eighteen quantities including COO order, after every entry point was routed through mpopf_args.

opf_model returned the recipe's raw handles again — the entry-point unification rewrote the function and dropped the instantiate that 5e38e1e had added, so solution(result, v) threw the symbolic-range ArgumentError once more. test_solution_handles caught its own regression, which is exactly what it was written for; the suite run that caught it is the gate for this branch.

…needs Two faults, both mine and both caught by CI doing its job. The NLPModels entry in [extras] carried a fabricated UUID — the first group was right and the tail was written from memory, so every test leg failed at resolution with 'expected package to be registered'. Corrected to the registered UUID and verified against the registry. The registered ExaModels 0.11.2 predates the recipe API: the repo carries that version number but was last registered before ExaCore(nargs = ...) existed, so no leg can resolve a usable ExaModels from the registry, and the CUDA leg additionally needs the deferred-collect residence fix that is part of madsuite-org/ExaModels.jl#308. Pinned by [sources] to the #308 tip SHA in both the package and docs projects — verified to resolve and carry the API. The pin is temporary scaffolding: it comes out when ExaModels next registers, and this package cannot itself be registered until then, which the PR notes.

sshin23 · 2026-08-13T12:54:48Z

The first CI round failed for two reasons, both now fixed in 579328b:

The NLPModels entry added to [extras] carried a wrong UUID (expected package to be registered). Corrected and verified against the registry.
The registered ExaModels 0.11.2 predates the recipe API (ExaCore(nargs = ...) does not exist there — the repo carries the 0.11.2 version number but was last registered before it), and the CUDA leg additionally needs the deferred-collect residence fix from ExaModelsC: emit several models into one library ExaModels.jl#308. Both projects now pin ExaModels by [sources] to the #308 tip SHA.

The pin is temporary scaffolding: it should come out when ExaModels next registers a release containing #308, and this package cannot itself be registered until that release exists — the ExaModels = "0.11" compat entry is only satisfiable from the registry by a version that lacks the API this PR uses.

@autodocs

The mpopf_args_default docstring cross-references it, docs/src/core.md is an @autodocs page, and Documenter's cross-reference check is strict — an @ref to an undocumented symbol terminates the build. It deserved the docstring regardless: it is the documented route to a different curve. Verified by building the docs locally before pushing (exit 0, demos executed, no unresolved references) — the second docs-only CI round is the one that should not have needed CI to find.

codecov · 2026-08-13T13:45:48Z

Codecov Report

❌ Patch coverage is 78.15385% with 71 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.63%. Comparing base (88aaada) to head (d41c3ac).

Files with missing lines	Patch %	Lines
src/mpopf.jl	70.32%	54 Missing ⚠️
src/opf.jl	91.47%	11 Missing ⚠️
src/constraint.jl	0.00%	4 Missing ⚠️
src/ExaModelsPower.jl	75.00%	1 Missing ⚠️
src/parser.jl	83.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #57      +/-   ##
==========================================
- Coverage   83.67%   80.63%   -3.05%     
==========================================
  Files           9        8       -1     
  Lines        1354     1477     +123     
==========================================
+ Hits         1133     1191      +58     
- Misses        221      286      +65

Flag	Coverage Δ
cpu	`25.27% <77.57%> (+1.75%)`	⬆️
cuda	`25.27% <77.57%> (+1.75%)`	⬆️
goc3	`58.34% <2.18%> (-5.01%)`	⬇️
nothing	`25.60% <78.88%> (+2.08%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sshin23 · 2026-08-13T14:08:54Z

Do not merge (ExaModels currently pinned for testing)

The formulation decides which methods run, so as a Symbol it is a run-time value and every call through it is type-unstable. As an instance it is part of the type the compiler sees. Instances are now the defaults and what the docs teach; the Symbols still work everywhere, resolved by opf_form at the boundary, documented as the compatibility spelling with the instability named as its price. OPFForm, Polar, Rect and DC are exported and documented. Verified: the instance and Symbol spellings build identical models, the dcopf aliases and DC multi-period still work, and the static equivalence set is unchanged, 36/36.

form is an OPFForm instance and nothing else: opf_form is gone, and every entry point declares form::OPFForm = Polar(). A Symbol now fails at the call — TypeError: in keyword argument form, expected OPFForm, got a value of type Symbol — rather than being resolved at run time, which is the point: the formulation decides which methods run, so as a Symbol it made every call through it type-unstable. Version 0.4.0: this breaks form = :polar / :rect / :dc at every entry point, which is the whole of the API change. Docstrings, the README, the docs demos and the test form tables all use instances. Verified: static equivalence unchanged at 36/36 against the pre-recipe constructors, the Symbol spelling confirmed refused, and the CPU suite 604/604.

sshin23 · 2026-08-13T15:13:19Z

Breaking change added (0.4.0): the formulation is passed as a type instance, not a Symbol.

opf_model(file; form = Rect())      # was form = :rect
mpopf_model(file, curve; form = DC())

OPFForm, Polar, Rect and DC are exported. opf_form is gone and every entry point declares form::OPFForm = Polar(), so a Symbol fails at the call with TypeError: in keyword argument form, expected OPFForm, got a value of type Symbol.

Rationale: the formulation decides which methods run, so as a Symbol it is a run-time value and every call through it is type-unstable; as an instance it is part of the type the compiler sees. Version bumped to 0.4.0 accordingly — this breaks form = :polar / :rect / :dc at every entry point, which is the whole of the API change. Docstrings, README, docs demos and the test form tables all use instances.

Gates: static equivalence unchanged at 36/36 against the pre-recipe constructors, CPU suite 604/604 locally, Symbol refusal verified.

Both from review. The demo passed kkt_system = SparseCondensedKKTSystem and linear_solver = CUDSSSolver to madnlp. Neither is needed: CUDSS is loaded, so MadNLP's CUDA extension activates and picks the GPU path itself. A tutorial should show the shortest call that works. And the generated .md files are checked in, so my local docs build committed this machine's precompile output into them — 'Precompiling packages... 28902.2 ms CUDATools' and similar. That is environment noise, not documentation. The .md files are restored to their committed output with only the two source-driven edits applied: the formulation instances, and the simplified madnlp call.

sshin23 · 2026-08-13T16:34:09Z

Both addressed in d41c3ac:

kkt_system / linear_solver — dropped from docs/src/opf_demo.jl; the call is madnlp(model; tol=1e-6) again. CUDSS is loaded in the demo, so MadNLP's CUDA extension activates and selects the GPU path without being told.
The precompile block — that was environment noise from my own docs build. The .md files are generated but checked in, so regenerating them locally committed this machine's Precompiling packages... output. Restored to the committed output with only the two source-driven edits applied (formulation instances, and the simplified madnlp call), so the diff to docs/src/*.md is now 2 lines rather than 78.

sshin23 added 11 commits August 11, 2026 23:07

sshin23 added 2 commits August 13, 2026 10:30

sshin23 changed the title ~~Write every model as a recipe: one body per class, AOT-compilable, DC and DC-multi-period included~~ Write every model as a recipe: one body per class, AOT-compilable, DC and DC-multi-period included (breaking: form is a type, 0.4.0) Aug 13, 2026

sshin23 commented Aug 13, 2026

View reviewed changes

Comment thread docs/src/opf_demo.md Outdated

Comment thread docs/src/opf_demo.md Outdated

sshin23 merged commit 5f64c18 into main Aug 14, 2026
5 of 7 checks passed

Navigation Menu

Uh oh!

Write every model as a recipe: one body per class, AOT-compilable, DC and DC-multi-period included (breaking: form is a type, 0.4.0) - #57

Write every model as a recipe: one body per class, AOT-compilable, DC and DC-multi-period included (breaking: form is a type, 0.4.0)#57
sshin23 merged 15 commits into
mainfrom
ohm/examodelsc

sshin23 commented Aug 13, 2026

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

codecov Bot commented Aug 13, 2026 •

edited

Loading

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

Uh oh!

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

sshin23 commented Aug 13, 2026

What this does

One body per model class

Nothing about the models changes

Inference fixes that AOT forced and the package keeps

Tests

Depends on

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

codecov Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

Uh oh!

Uh oh!

sshin23 commented Aug 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

codecov Bot commented Aug 13, 2026 •

edited

Loading