Armor core libraries with symbol visibility

[Copilot]

Adds C++ symbol visibility control (hidden-by-default) to all Phlex shared libraries, implemented de novo on current main.

Symbol visibility mechanism

Modules/private/PhlexSymbolVisibility.cmake provides a single function:

phlex_apply_symbol_visibility(phlex_core)

This generates an export header via GenerateExportHeader into a centralized ${PROJECT_BINARY_DIR}/include directory, sets CXX_VISIBILITY_PRESET hidden + VISIBILITY_INLINES_HIDDEN ON, wires up include paths, and installs the generated header for downstream consumers.

Libraries armored

phlex_apply_symbol_visibility() applied to all 5 shared libraries: phlex_core, phlex_model, phlex_utilities, phlex_configuration_internal, run_phlex.

Export macro coverage

~45 headers decorated across all libraries — every class and free function with a .cpp implementation gets the appropriate export macro. Template-only and header-only code left unmarked (instantiated in consumer TUs).

#include "phlex_core_export.hpp"
// ...
class phlex_core_EXPORT framework_graph { ... };
phlex_core_EXPORT std::size_t port_index_for(...);

Changes

• New Modules/private/PhlexSymbolVisibility.cmake module with phlex_apply_symbol_visibility() function
• 6 CMakeLists.txt files updated (root + 5 library targets)
• ~42 header files annotated with export macros across all libraries
• Export headers generated into centralized ${PROJECT_BINARY_DIR}/include and installed for downstream consumers
• Consistent friend + export macro usage (no redundant namespace-scope redeclarations)

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[greenc-FNAL]

@phlexbot format

[github-actions]

Automatic markdownlint fixes pushed (commit 3299ec5).
⚠️ Note: Some issues may require manual review and fixing.

[github-actions]

No automatic jsonnetfmt fixes were necessary.

[github-actions]

Automatic clang-format fixes pushed (commit 943ef76).
⚠️ Note: Some issues may require manual review and fixing.

[github-actions]

Automatic cmake-format fixes pushed (commit ccb09dd).
⚠️ Note: Some issues may require manual review and fixing.

[Copilot]

Pull request overview

Updates the symbol-visibility feature branch to main and completes hidden-by-default symbol export coverage across Phlex shared libraries, while also modernizing the driver/source plumbing (data-cell indexing, providers, product queries) and refreshing tests/examples accordingly.

Changes:

• Apply GenerateExportHeader + hidden visibility presets and expand export-macro coverage across all shared libraries.
• Replace legacy level-id/label APIs with data_cell_index + product_query (and update tests/configs/plugins accordingly).
• Add/refresh Python/plugin tooling and CI hygiene (ruff/mypy config, SARIF helper, workflow tweaks).

Reviewed changes

Copilot reviewed 287 out of 289 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
test/yielding_driver.cpp	Update test to data_cell_index driver model
test/vector_of_abstract_types.cpp	Update graph test to providers + "_in(layer)" queries
test/utilities/CMakeLists.txt	Migrate tests to cet_test
test/specified_label.cpp	Remove tests for deleted specified_label API
test/replicated.cpp	Narrow namespace usage for updated utilities
test/python/verify.py	Add Python observer helper for tests
test/python/test_phlex.py	Update Python import expectations / docs
test/python/sumit.py	Add numpy-based Python algorithm test module
test/python/source.cpp	Add C++ providers for Python tests
test/python/registry.py	Remove old Python registry glue
test/python/reducer.py	Add Python reduction workflow test module
test/python/pyvec.jsonnet	Add Python/numpy config fixture
test/python/pyreduce.jsonnet	Add Python reduction config fixture
test/python/pyfailure.jsonnet	Add intentional Python config failure fixture
test/python/pyconfig.jsonnet	Add “all config types” Python fixture
test/python/pyadd.jsonnet	Update Python add fixture to new driver/sources layout
test/python/phlexpy.py	Update cppyy bootstrap + pythonization docs
test/python/failing_test_wrap.sh	Add wrapper for expected-failure tests
test/python/driver.cpp	Remove legacy C++ driver for Python tests
test/python/all_config.py	Add Python-side configuration consumer
test/python/adder.py	Replace cppyy-defined add with pure Python + registration
test/provider_test.cpp	New provider-focused integration test
test/product_store.cpp	Extend product_store tests for size/empty & id changes
test/product_query.cpp	New tests for product_query + _in literal
test/plugins/source.cpp	Remove legacy plugin source
test/plugins/output.cpp	Update plugin registration macro / namespaces
test/plugins/module.cpp	Update plugin to product_query + new macro
test/plugins/ij_source.cpp	New provider plugin for i/j inputs
test/plugins/add.jsonnet	Update plugin job config to new driver/sources layout
test/plugins/CMakeLists.txt	Add provider plugin + migrate job test to cet_test
test/multiple_function_registration.cpp	Update multi-transform test to provider inputs / new hierarchy
test/mock-workflow/three_tuple_algorithm.cpp	Update workflow module registration + input index type
test/mock-workflow/source.cpp	Remove legacy mock-workflow source
test/mock-workflow/pd_fast_sim.cpp	Update plugin registration macro
test/mock-workflow/mock-workflow.jsonnet	Update workflow config to driver+provider layout
test/mock-workflow/largeant.cpp	Update plugin registration macro
test/mock-workflow/ion_and_scint.cpp	Update plugin registration macro
test/mock-workflow/id_provider.cpp	New provider plugin for “id” product
test/mock-workflow/event_product.libsonnet	Add helper for event-layer product specs
test/mock-workflow/algorithm.hpp	Update algorithm test helper to use product_query inputs
test/mock-workflow/SinglesGen.libsonnet	Update config keys (cpp) + product specs
test/mock-workflow/MC_truth_algorithm.cpp	Update input index type
test/mock-workflow/G4Stage2.libsonnet	Update config keys + product specs
test/mock-workflow/G4Stage1.libsonnet	Update config keys + product specs
test/mock-workflow/CMakeLists.txt	Replace source plugin + migrate test execution to cet_test
test/memory-checks/many_events.cpp	Update memory-check driver/source plumbing
test/memory-checks/CMakeLists.txt	Migrate test to cet_test + link layer_generator
test/max-parallelism/provide_parallelism.cpp	Add provider plugin for parallelism value
test/max-parallelism/check_parallelism_default.jsonnet.in	Update config to driver+provider layout
test/max-parallelism/check_parallelism.cpp	Remove embedded source; update to product_query input
test/function_registration.cpp	Update test inputs to provider registration
test/framework_graph.cpp	Add new graph behavior tests (exceptions, single-thread progress)
test/form/test_helpers.hpp	Add helpers for FORM test type map
test/form/form_test.jsonnet	Add form-module job config
test/form/data_products/CMakeLists.txt	Clean up ROOT dictionary linkage
test/form/CMakeLists.txt	Migrate FORM tests to cet_test + add job test
test/filter_impl.cpp	Add test for data_map completeness behavior
test/demo-giantdata/waveforms.cpp	Remove logging hooks; simplify dtors
test/demo-giantdata/waveform_generator_input.cpp	Remove logging hooks; simplify dtors
test/demo-giantdata/waveform_generator.hpp	Remove unused members/includes
test/demo-giantdata/waveform_generator.cpp	Remove logging hooks; simplify ctor/dtor
test/demo-giantdata/user_algorithms.hpp	Simplify APIs (drop unused id args)
test/demo-giantdata/user_algorithms.cpp	Remove logging hooks; simplify clamp/accumulate
test/demo-giantdata/CMakeLists.txt	Convert demo to cet_test and link generator
test/data_cell_index.cpp	Replace level-id tests with data-cell index hashing test
test/configuration.cpp	Add configuration error and product_query parsing tests
test/class_registration.cpp	Update test inputs to providers + product_query
test/cached_product_stores.cpp	Remove tests for deleted cache helper
test/cached_execution_source.hpp	Remove legacy cached execution source
test/cached_execution.cpp	Update cached execution test to providers + layer generator
test/benchmarks/verify_even_fibonacci_numbers.cpp	Update plugin macro + product_query input parsing
test/benchmarks/verify_difference.cpp	Update config parsing to product_query inputs
test/benchmarks/read_index.cpp	Update plugin macro + config parsing
test/benchmarks/read_id.cpp	Update id type + input query
test/benchmarks/plus_one.cpp	Update plugin macro + input layer
test/benchmarks/plus_101.cpp	Update plugin macro + input layer
test/benchmarks/last_index.cpp	Update id type + input layer
test/benchmarks/benchmarks_source.cpp	Remove legacy benchmark source
test/benchmarks/benchmarks_provider.cpp	Add provider plugin for benchmark id product
test/benchmarks/benchmark-09.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-08.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-07.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-06.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-05.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-04.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-03.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-02.jsonnet	Update config to driver+provider layout
test/benchmarks/benchmark-01.jsonnet	Update config to driver+provider layout
test/benchmarks/accept_fibonacci_numbers.cpp	Update plugin macro + product_query input parsing
test/benchmarks/accept_even_numbers.cpp	Update plugin macro + product_query input parsing
test/benchmarks/accept_even_ids.cpp	Update id type + config parsing
test/benchmarks/CMakeLists.txt	Replace source plugin with provider + migrate job tests
test/adjust_config.cpp	Add tests for module config normalization (cpp/py)
scripts/sarif-alerts.py	Add SARIF alert printing tool
scripts/export_llvm_lcov.py	Add docstrings for script entrypoints
scripts/create_coverage_symlinks.py	Add docstrings for coverage symlink helper
pyproject.toml	Introduce ruff+mypy configuration
plugins/python/src/wrap.hpp	Add Python registration wrappers + GIL RAII
plugins/python/CMakeLists.txt	Add Python plugin build + optional NumPy support
plugins/generate_layers.cpp	Add configurable layer-generator driver plugin
plugins/CMakeLists.txt	Add core plugins build/install
phlex/utilities/stripped_name.hpp	Remove deleted utility header
phlex/utilities/stripped_name.cpp	Remove deleted utility implementation
phlex/utilities/async_driver.hpp	Switch to std::jthread + propagate driver exceptions
phlex/utilities/CMakeLists.txt	Migrate utilities build/install to cetmodules + visibility
phlex/module.hpp	Replace old registration macro with shared plugin macro helper
phlex/model/qualified_name.hpp	Remove deprecated qualified-name API
phlex/model/qualified_name.cpp	Remove deprecated qualified-name implementation
phlex/model/products.hpp	Switch to product_specification + export + add size/empty
phlex/model/products.cpp	Implement size/empty
phlex/model/product_specification.hpp	Add replacement for qualified-name outputs (with type id)
phlex/model/product_specification.cpp	Implement product_specification + helpers
phlex/model/product_matcher.hpp	Rename level→layer terminology + export
phlex/model/product_matcher.cpp	Update matcher pattern strings for layer terminology
phlex/model/level_id.hpp	Remove legacy level-id API
phlex/model/level_counter.hpp	Remove legacy level-counter API
phlex/model/level_counter.cpp	Remove legacy level-counter implementation
phlex/model/handle.hpp	Update handle to carry data_cell_index
phlex/model/fwd.hpp	Add data-cell index fwd decls + update model fwd set
phlex/model/data_layer_hierarchy.hpp	Rename/replace level hierarchy with data-layer hierarchy
phlex/model/data_cell_index.hpp	Add new data-cell index type
phlex/model/data_cell_counter.hpp	Add new counters for flush accounting
phlex/model/data_cell_counter.cpp	Implement new counters
phlex/model/CMakeLists.txt	Migrate model build/install to cetmodules + visibility
phlex/metaprogramming/type_deduction.hpp	Expand traits/concepts (atomic/container/tuple helpers)
phlex/metaprogramming/CMakeLists.txt	Migrate metaprogramming install/export to cetmodules
phlex/graph/serializer_node.hpp	Include-order tweak for export header
phlex/graph/serial_node.hpp	Include-order tweak for export header
phlex/graph/CMakeLists.txt	Migrate graph build/install to cetmodules + visibility
phlex/driver.hpp	Add unified driver plugin registration + create_next helper
phlex/detail/plugin_macros.hpp	Add shared plugin registration macro machinery
phlex/core/upstream_predicates.hpp	Rename when to experimental_when
phlex/core/store_counters.cpp	Update flush/store accounting to data-cell ids
phlex/core/specified_label.hpp	Remove deprecated specified-label API
phlex/core/specified_label.cpp	Remove deprecated specified-label implementation
phlex/core/registration_api.cpp	Rename when to experimental_when in API
phlex/core/registrar.hpp	Export helper used across shared-library boundary
phlex/core/products_consumer.hpp	Switch inputs to product_query + export
phlex/core/products_consumer.cpp	Switch inputs to product_query
phlex/core/product_query.hpp	Add product_query + _in(layer) literal + type population
phlex/core/product_query.cpp	Implement product_query + literal error handling
phlex/core/node_catalog.hpp	Export node_catalog + add provider catalog
phlex/core/node_catalog.cpp	Add providers to execution_counts + throw on unknown name
phlex/core/multiplexer.hpp	Update multiplexer finalize model for providers
phlex/core/message_sender.hpp	Update sender to use data-layer hierarchy + data-cell ids
phlex/core/message_sender.cpp	Update to new hierarchy constructor
phlex/core/message.hpp	Switch to product_query ports + export required symbols
phlex/core/message.cpp	Update to data_cell_index and new error text
phlex/core/input_arguments.hpp	Switch retrievers to product_query + export duplicate check
phlex/core/input_arguments.cpp	Update duplicate detection types
phlex/core/graph_proxy.hpp	Add provide() to graph proxy + namespace adjustments
phlex/core/glue.hpp	Add provider registration + export name verification
phlex/core/fwd.hpp	Move framework_driver alias to phlex::framework_driver
phlex/core/filter.hpp	Export filter class
phlex/core/end_of_message.hpp	Update to new hierarchy + export
phlex/core/end_of_message.cpp	Update to new hierarchy types
phlex/core/edge_maker.hpp	Add provider edge support + export helper
phlex/core/edge_maker.cpp	Implement provider edge wiring
phlex/core/edge_creation_policy.hpp	Switch producer matching to product_query + include types
phlex/core/edge_creation_policy.cpp	Add type-aware producer matching + better diagnostics
phlex/core/detail/maybe_predicates.hpp	Export predicate extraction + rename config key
phlex/core/detail/maybe_predicates.cpp	Read experimental_when from config
phlex/core/detail/make_algorithm_name.hpp	Export algorithm-name helper
phlex/core/detail/filter_impl.hpp	Export filter maps + switch to product_query
phlex/core/detail/filter_impl.cpp	Fix completeness logic for sparse store vectors
phlex/core/declared_unfold.cpp	Update unfold generator to new data-cell API
phlex/core/declared_transform.cpp	Update ctor arg types + improve cached-store reporting
phlex/core/declared_provider.cpp	Add declared provider implementation
phlex/core/declared_predicate.hpp	Switch to product_query + data-cell hashes + export
phlex/core/declared_predicate.cpp	Switch ctor arg types
phlex/core/declared_output.hpp	Export declared_output
phlex/core/declared_observer.hpp	Switch to product_query + data-cell hashes + export
phlex/core/declared_observer.cpp	Switch cached-hash map key type
phlex/core/declared_fold.cpp	Switch ctor arg types
phlex/core/consumer.hpp	Export consumer base
phlex/core/concurrency.cpp	Move concurrency constants to phlex namespace
phlex/core/concepts.hpp	Add provider concept
phlex/core/cached_product_stores.hpp	Remove deprecated store-cache helper
phlex/configuration.cpp	Add product_query JSON conversion + keys() helper
phlex/concurrency.hpp	Export concurrency + move to phlex namespace
phlex/app/version.hpp	Export version() across shared library boundary
phlex/app/run.hpp	Export run() across shared library boundary
phlex/app/run.cpp	Update config schema (driver/modules/sources) + better errors
phlex/app/phlex.cpp	Catch+print top-level exceptions in CLI
phlex/app/load_module.hpp	Export app API + add adjust_config + new loader functions
phlex/app/CMakeLists.txt	Migrate app build to cetmodules + symbol visibility
phlex/CMakeLists.txt	Register export set + split configuration lib + install headers
form/storage/CMakeLists.txt	Formatting / target definition cleanup
form/root_storage/root_ttree_container.cpp	Fix write semantics on destruction + add guard
form/root_storage/CMakeLists.txt	Formatting / target definition cleanup
form/mock_phlex/phlex_toy_core.hpp	Remove mock-phlex shim (no longer used)
form/mock_phlex/phlex_toy_core.cpp	Remove mock-phlex shim implementation
form/mock_phlex/phlex_toy_config.hpp	Remove mock-phlex config shim
form/mock_phlex/phlex_toy_config.cpp	Remove mock-phlex config shim implementation
form/mock_phlex/CMakeLists.txt	Remove mock-phlex library
form/form/form.hpp	Replace mock-phlex types with FORM-local types
form/README.md	Minor markdown cleanup
form/CMakeLists.txt	Enable PIC + add form_module plugin
codecov.yml	Raise coverage thresholds
README-dev.md	Remove developer notes file
Modules/private/PhlexTargetUtils.cmake	CMake formatting/cleanup
Modules/private/PhlexSymbolVisibility.cmake	CMake formatting/cleanup for visibility helper
.github/workflows/python-fix.yaml	Add workflow to auto-fix Python formatting/lint
.github/workflows/add-issues.yaml	Pin action by SHA + permissions hardening
.github/dependabot.yml	Add Dependabot for GitHub Actions
.github/actions/get-pr-info/action.yaml	Add composite action to resolve PR refs from comments
.github/actions/generate-build-matrix/action.yaml	Add composite action for dynamic build matrix
.github/actions/detect-relevant-changes/action.yaml	Add python file patterns + ref validation
.github/actions/detect-act-env/action.yaml	Add act-environment detection helper
.github/actions/configure-cmake/action.yaml	Refactor CMake configure args + enable color forcing
.github/actions/build-cmake/action.yaml	Force colored output
.github/actionlint.yaml	Add actionlint configuration scaffolding
.github/CodeQL-README.md	Markdown/formatting improvements
.github/CODEOWNERS	Add CODEOWNERS enforcement baseline
.flake8	Remove flake8 config (ruff replaces it)

[greenc-FNAL]

@copilot I think for maximum clarity, it would be better to reset this branch to upstream/main and then re-do the work following the current philosophy, library names, etc. Otherwise this looks like a huge body of work that would be way to much for a human to review.

[Copilot]

@copilot I think for maximum clarity, it would be better to reset this branch to upstream/main and then re-do the work following the current philosophy, library names, etc. Otherwise this looks like a...

Done. In commit aeae5d2 I reset the branch to current main, then in c4cb573 I re-implemented the symbol visibility changes cleanly on top. The diff is now purely the visibility work — 51 files changed (1 new CMake module, 7 CMakeLists.txt edits, 43 header files with export macros). No merge noise.

[Copilot]

Pull request overview

Copilot reviewed 51 out of 51 changed files in this pull request and generated 3 comments.

[greenc-FNAL]

@copilot apply changes based on the comments in this thread

[Copilot]

@copilot apply changes based on the comments in this thread

All three review comments addressed in 017903d:

• Export headers now generate into ${PROJECT_BINARY_DIR}/include (centralized)
• Export headers are installed via install(FILES ...)
• operator<< friend/free declaration inconsistency fixed in data_cell_index.hpp

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main     #322      +/-   ##
==========================================
+ Coverage   86.51%   86.55%   +0.04%     
==========================================
  Files         119      119              
  Lines        2403     2403              
  Branches      387      387              
==========================================
+ Hits         2079     2080       +1     
- Misses        206      207       +1     
+ Partials      118      116       -2     

Flag	Coverage Δ
unittests	86.55% <ø> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
phlex/configuration.hpp	58.33% <ø> (ø)
phlex/core/declared_fold.hpp	95.31% <ø> (ø)
phlex/core/declared_observer.hpp	100.00% <ø> (ø)
phlex/core/declared_output.hpp	100.00% <ø> (ø)
phlex/core/declared_predicate.hpp	94.73% <ø> (ø)
phlex/core/declared_provider.hpp	100.00% <ø> (ø)
phlex/core/declared_transform.hpp	88.57% <ø> (ø)
phlex/core/declared_unfold.hpp	96.42% <ø> (ø)
phlex/core/detail/filter_impl.hpp	100.00% <ø> (ø)
phlex/core/edge_creation_policy.hpp	80.00% <ø> (ø)
... and 22 more

... and 5 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 103c08c...ebfc86b. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[greenc-FNAL]

@phlexbot clang-fix

[greenc-FNAL]

@phlexbot format

[github-actions]

No automatic markdownlint fixes were necessary.

[github-actions]

Automatic cmake-format fixes pushed (commit 8ba091a).
⚠️ Note: Some issues may require manual review and fixing.

[github-actions]

No automatic jsonnetfmt fixes were necessary.

[github-actions]

No automatic clang-format fixes were necessary.

[greenc-FNAL]

@copilot This PR (branch copilot/update-branch-with-main) is once more behind main, with extensive conflicts. Please evaluate whether rebasing and updating with respect to the original goals is feasible, or whether it would be more prudent to rebase preserving only the infrastructure changes, and implement the original prompt's goals in the C++ code de novo.

[Copilot]

@copilot This PR (branch copilot/update-branch-with-main) is once more behind main, with extensive conflicts. Please evaluate whether rebasing and ...

Rebasing wasn't feasible — 31 new commits on main touch 28 of the files we modified, including a complete rewrite of product_query (#267), removal of the phlex_graph library, new index_router/repeater_node classes, and clang-tidy naming changes throughout.

Instead I reset to main in 58472b7 and re-implemented the symbol visibility de novo in 77b19ca. The diff is purely the visibility work: 49 files changed (1 new CMake module, 6 CMakeLists.txt edits, 42 header files with export macros). Covers all 5 current shared libraries — phlex_graph no longer exists on main.

[greenc-FNAL]

@phlexbot format

[github-actions]

No automatic format fixes were necessary.

[greenc-FNAL]

@phlexbot format

[github-actions]

No automatic format fixes were necessary.

[greenc-FNAL]

@phlexbot clang-fix

[github-actions]

Automatic clang-format fixes pushed (commit ebfc86b).
⚠️ Note: Some issues may require manual review and fixing.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 49 changed files in this pull request and generated 4 comments.

[Copilot]

The exported UDL declarations use std::size_t but this header does not include <cstddef>. Add an explicit <cstddef> include to avoid depending on transitive headers.

[Copilot]

load_module.hpp uses std::string in exported declarations but does not include <string> directly, which makes it dependent on transitive includes from other headers. Add #include <string> here to keep the header self-contained.

[Copilot]

hashing.hpp uses std::size_t in the exported function declarations but does not include <cstddef> (it currently relies on transitive includes). To keep this header self-contained/portable, add an explicit #include <cstddef> here.

[Copilot]

This header uses std::strong_ordering in exported friend declarations, but does not include <compare>. Relying on transitive includes can break builds on other standard library implementations; consider adding #include <compare> to make the header self-contained.