Feat/schema improvements and optimization - #95
Open
carlosengutierrez wants to merge 7 commits into
Open
Conversation
added 7 commits
August 10, 2026 01:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds parameter optimization to Neuro-Workflow: a search loop that runs an existing workflow
repeatedly, tunes declared parameters and reports how close each run lands to a declared target.
A workflow is a DAG that runs once; a search is a loop that runs it many times, so the loop sits
outside the graph rather than becoming a node inside it.
What is here
src/neuroworkflow/optimization/— the engine: dotted addressing (Node.parameter[.key]totune,
Node.output_port[.key]to measure), the spec, a file-based ledger, and optimizer backends(random plus Optuna's CMA-ES, TPE, NSGA-II/III). Single-objective backends refuse a
multi-objective spec rather than inventing weights between objectives.
NW_Optimizationnode — declares how to search: algorithm, budget, seed. No ports and noprocess steps, so a workflow containing it executes exactly as it would without it. Its presence
on the canvas is the signal for the generator to emit an optimization run instead of a single
execution. Replaces
JointOptimizationNode, which was an in-graph grid search.NW_SimConfigrebuilds the SONATA network only whensomething structural changed, so a search does not rebuild it once per trial. Nodes opt in with
REUSABLE_PATHS; the engine itself stays domain-free.NW_Analysis—firing_rate_hzandisi_statsper population, so a targethas something to be compared against.
unitandmeasuresonParameterDefinition;optimization_rangeaccepts aper-key dict for dict-valued parameters. Each node instance now carries its own
NODE_DEFINITIONcopy, so markingexcoptimizable no longer affectsinh.Reporting across objectives in different units
A miss in Hz and a miss in ms cannot be added. Each objective reports its own miss in its own unit,
and the single comparable figure is the worst objective's miss divided by the width of its target
range — printed as
furthest from target: probe_isi, 1.4x its target rangeand stored astarget_ranges_off. Pareto dominance is scale-free and NSGA-II rescales internally, so thisaffects reporting and ranking only; what the optimizer is told stays raw and per-objective.
Docs and examples
docs/OPTIMIZATION.mdfor how the engine works,docs/OPTIMIZATION_GUI_HANDOFF.mdfor the GUIhalf. Five notebooks under
notebooks/, two of which show what the code generator should emit.Not done, and needed before the GUI can use this
optunaandcmaesare not in the nest kernel image.NW_Optimizationdefaults tocmaes,so the first optimization generated in the GUI fails on import until one line is added at
Dockerfile.nest:128. Left out of this PR because it forces a rebuild of the nest image; it iswritten up as step 1 of the handoff document.
Testing
Single- and multi-objective searches run end to end in Jupyter against real NEST/BMTK, and with toy
nodes for the engine paths. GUI copies under
codes/are synced and byte-identical tosrc/.