Benchmark results

[jaclark5]

No description provided.

[Copilot]

Pull request overview

This PR introduces a BenchmarkResults loading/filtering API (including lazy FEMap generation), adds CI-oriented validation helpers for benchmark submissions, and updates generated submission metadata + examples/tests to support the new schema (notably adding calculation_type to submission YAMLs).

Changes:

• Add openfe_benchmarks.results API (BenchmarkResults, get_benchmark_results, filter_results) plus FEMap helpers for absolute/relative results.
• Add CI-focused validation utilities (_validation.py) and new pytest coverage for results loading/filtering and CI validation strategy.
• Update metadata generation and existing submission YAMLs to include calculation_type, plus refresh plotting/example scripts and docs notebook.

Reviewed changes

Copilot reviewed 21 out of 26 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
openfe_benchmarks/tests/test_results_validation.py	Adds CI-style validation/performance tests for YAML validation, changed-file detection, and sampling.
openfe_benchmarks/tests/test_benchmark_results.py	Adds tests for loading/filtering/FEMap behavior and error handling in the new results API.
openfe_benchmarks/scripts/prepare_metadata_submission.py	Updates submission YAML generation (tags, calc type, network mode) and metadata extraction logic.
openfe_benchmarks/scripts/_results_utils.py	Adds absolute-results FEMap builder alongside existing relative-results FEMap builder.
openfe_benchmarks/scripts/_no_test_example_rbfe_asfe_submission.py	Updates example defaults (notably tags handling) and notes ASFE compatibility.
openfe_benchmarks/scripts/_no_test_example_plot_asfe.py	New example showing ASFE plotting via the new results API.
openfe_benchmarks/scripts/_no_test_example_mutlinetwork_rbfe_submission.py	Updates example placeholder network keys and tag defaults.
openfe_benchmarks/scripts/_example_plot_rbfe.py	Refactors RBFE plotting example to use BenchmarkResults API and lazy FEMaps.
openfe_benchmarks/results/2026-08-06-openff-2.3.0-solvation_set_freesolv/submission.yaml	Adds new ASFE submission YAML including calculation_type.
openfe_benchmarks/results/2026-08-05-openff3.0.0-alpha1b_tip3p-jacs/submission.yaml	Adds calculation_type: rbfe.
openfe_benchmarks/results/2026-08-04-openff3.0.0-alpha1b_opc3-jacs/submission.yaml	Adds calculation_type: rbfe and adjusts tags.
openfe_benchmarks/results/2026-06-22-tyk2-alchemicalarchive-test/submission.yaml	Adds calculation_type: rbfe and adjusts tags.
openfe_benchmarks/results/2026-03-18-openmm-840-qa-testing/submission.yaml	Adds calculation_type: rbfe.
openfe_benchmarks/results/2026-02-12_sage_230_jacs_set/submission.yaml	Adds calculation_type: rbfe.
openfe_benchmarks/results/2026_08_05_openff-3.0.0-alpha0_opc3_jacs/submission.yaml	Adds calculation_type: rbfe.
openfe_benchmarks/results/2026_08_05_ff14sb_openff-3.0.0-alpha1b_tip3p_jacs_tyk2_thrombin/submission.yaml	Adds calculation_type: rbfe.
openfe_benchmarks/results/_validation.py	Adds CI helper functions for fast YAML validation, changed submission detection, and sampling.
openfe_benchmarks/results/_benchmark_results.py	Adds BenchmarkResults dataclass, factory loader, filtering, and lazy FEMap properties.
openfe_benchmarks/results/init.py	Exposes the new results API via package exports.
examples/4_benchmark_result_plot.ipynb	Adds a notebook demonstrating loading, filtering, FEMap generation, and plotting.
devtools/conda-envs/environment.yml	Updates environment constraints/comments related to dependencies.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jaclark5]

This cannot be uncommented because of a conflict with the openfe version needed to deserialize the networks from alchemiscale. After a stack update this can be uncommented.