Skip to content

Add params schema support to ProcessNode and align legacy _from_scriptconfig#7

Open
Erotemic wants to merge 4 commits intomainfrom
codex/add-params-class-variable-to-processnode
Open

Add params schema support to ProcessNode and align legacy _from_scriptconfig#7
Erotemic wants to merge 4 commits intomainfrom
codex/add-params-class-variable-to-processnode

Conversation

@Erotemic
Copy link
Collaborator

Motivation

  • Allow ProcessNode subclasses to declare a scriptconfig-style schema via a params class variable so path/parameter groups can be derived without factory helpers.
  • Preserve backwards compatibility by keeping the EXPERIMENTAL _from_scriptconfig helper while making it follow the same derivation rules as the new params mechanism.
  • Make grouping/tag rules explicit (in/out/algo/perf/primary) and handle defaults/warnings consistently when deriving groups from a schema.

Description

  • Introduce an optional class variable params on ProcessNode and merge derived groups into instance in_paths, out_paths, algo_params, perf_params, and primary_out_key during __init__ only where values are missing so explicit ctor/classvars take precedence.
  • Add a shared helper ProcessNode._derive_groups_from_params_spec(params_spec) that accepts a scriptconfig.DataConfig class or instance, or a plain dict, and returns (in_paths_set, out_paths_dict, algo_params_dict, perf_params_dict, primary_out_key) while implementing the following rules: no tags => algo param, raise ValueError on conflicting bucket tags, ignore (with warnings.warn) defaults for in_paths, use non-empty string defaults for out_paths, and include instance-provided defaults when params is an instance.
  • Update the legacy @classmethod _from_scriptconfig to call the shared helper and construct a ProcessNode using the same derivation rules while keeping it marked EXPERIMENTAL and treating it as a compatibility helper.
  • Add doctest-style examples demonstrating: untagged param becomes an algo default, in_path default triggers a warning and is ignored, explicit out_paths on the node override schema defaults, and parity between params-based derivation and _from_scriptconfig.
  • Minor plumbing: import warnings, ensure safe merging when in_paths/out_paths/algo_params/perf_params may be None, and preserve existing behavior of self.config and configure().

Testing

  • No automated tests were executed as part of this change (doctest examples were added to pipeline.py but not run).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant