diff --git a/.issueflows/01-current-issues/issue510_plan.md b/.issueflows/01-current-issues/issue510_plan.md index e09e822e..9b89414a 100644 --- a/.issueflows/01-current-issues/issue510_plan.md +++ b/.issueflows/01-current-issues/issue510_plan.md @@ -95,4 +95,9 @@ Prefer **three stacked PRs** on `510-…`, all closing #510 only when C lands (o ## Open questions -- None — ready for `/iflow-start` (Milestone A first). +- None for A/B (shipped). +- **Pin gate (C):** exact `cellpycore==` for the v2 release waits on + [cellpy/cellpy-core#136](https://github.com/cellpy/cellpy-core/issues/136) + + a new core PyPI release, then re-pin (see + `04-designs-and-guides/v2-cellpycore-pin-gate.md`). Do not treat `==0.2.1` + as the V2-15 pin. diff --git a/.issueflows/01-current-issues/issue510_status.md b/.issueflows/01-current-issues/issue510_status.md index 0f99404f..870a1f27 100644 --- a/.issueflows/01-current-issues/issue510_status.md +++ b/.issueflows/01-current-issues/issue510_status.md @@ -6,11 +6,21 @@ - Plan confirmed (2026-07-17): v9 zip-of-parquet + `.cellpy`, default write v9, native cols; three milestones A→B→C. - **Milestone A (V2-13)** — merged via PR #521. -- **Milestone B (V2-14)** — implemented: - - `cellpy/readers/cellpy_file/meta_archive.py` — `save_meta_archive` / `load_meta_archive` / `apply_meta_document` (core stubs stay stubs) - - v9 uses shared meta document; preserves campaign `test_id` columns on steps/summary - - Essential tests: `tests/test_meta_archive.py` + campaign v9 round-trip in `test_merge_campaign.py` +- **Milestone B (V2-14)** — merged via PR #522 (2026-07-17). +- GitHub issue #510 closed as completed when B landed (plan allowed closing after B and tracking C as release checklist). +- **Milestone C (V2-15) started** on branch `510-v2-release-discipline-milestone-c`: + - Draft user guide: `docs/getting_started/migration_v1_to_v2.md` (linked from getting-started toctree + HISTORY + data-structure note) ## Remaining work -- **C (V2-15):** exact `cellpycore==` pin + migration guide + release procedure +- **C (V2-15):** + - [x] v1→v2 migration guide (draft in tree; polish/link review still OK) + - [x] pin gate cleared: cellpy-core #136 → `cellpycore==0.2.2` on PyPI + - [x] `HeadersSummary` / `HeadersStepTable.test_id` parity + pipeline_smoke golden regen + - [ ] PR #523 CI green / merge + - [ ] follow `release-procedure.md` / cut `v2.0.0aN` when ready (tag timing may slip) + +## Pin gate — cellpy-core #136 + +Cleared (2026-07-17): core #136 shipped as `cellpycore==0.2.2`. cellpy pin + +header parity + pipeline_smoke golden updated on this branch. diff --git a/.issueflows/04-designs-and-guides/v2-cellpycore-pin-gate.md b/.issueflows/04-designs-and-guides/v2-cellpycore-pin-gate.md new file mode 100644 index 00000000..cacd28f0 --- /dev/null +++ b/.issueflows/04-designs-and-guides/v2-cellpycore-pin-gate.md @@ -0,0 +1,22 @@ +# v2 cellpycore pin gate (#510 Milestone C) + +**Context.** V2-15 wants an exact `cellpycore==` pin in the cellpy release +commit. cellpy already pins `cellpycore==0.2.1` (latest core PyPI as of +2026-07-15). + +**Decision.** Do not cut the v2.0 release pin (or claim V2-15 done) until +[cellpy/cellpy-core#136](https://github.com/cellpy/cellpy-core/issues/136) ships +in a **new** core release, then re-pin cellpy to that version. + +**Why.** #136 fixes legacy-bridge stripping of `test_id` on steps/summary and +the legacy-schema `merge_data` story. cellpy #507 works around the strip today; +releasing v2 against 0.2.1 would freeze the workaround as the supported path. + +**Sequence.** + +```text +core#136 fix → core bump/tag/PyPI → cellpy pin + UV_NO_SOURCES=1 uv lock + → essential green → v2.0.0aN / release checklist +``` + +**Refs.** cellpy #510 / #507 / #511; `.issueflows/04-designs-and-guides/release-procedure.md`. diff --git a/HISTORY.md b/HISTORY.md index 339f8ab0..34836b17 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -2,6 +2,14 @@ ## [Unreleased] +* Cellpy-file format v9 (#510, V2-13/14): default `save()` writes + zip-of-parquet + `meta.json` (``.cellpy``); `load()` still reads v4–v8 HDF5 + and sniffs v9. Full `TestMetaCollection` (+ units/limits) persists on v9; + campaign-merged multi-`test_id` objects round-trip. Escape: ``.h5`` / + `cellpy_file_format="hdf5"`. Cellpy-owned `save_meta_archive` / + `load_meta_archive` (core archive stubs stay stubs). User guide: + `docs/getting_started/migration_v1_to_v2.md`. + * `make_summary` gains `exclude_step_types` (#509, v2 theme V2-12): step-type prefixes (e.g. `["cv_"]`) whose per-cycle capacity contribution is subtracted from cycle-end charge/discharge capacities before derived columns — the diff --git a/cellpy/parameters/internal_settings.py b/cellpy/parameters/internal_settings.py index 3dfde24a..661a344d 100644 --- a/cellpy/parameters/internal_settings.py +++ b/cellpy/parameters/internal_settings.py @@ -456,6 +456,7 @@ class HeadersSummary(BaseHeaders): postfixes = ["gravimetric", "areal", "absolute"] cycle_index: str = "cycle_index" + test_id: str = "test_id" data_point: str = "data_point" test_time: str = "test_time" datetime: str = "date_time" @@ -554,6 +555,7 @@ class HeadersStepTable(BaseHeaders): """Headers used for the steps table (used as column headers for the steps pandas DataFrames)""" test: str = "test" + test_id: str = "test_id" ustep: str = "ustep" cycle: str = "cycle" step: str = "step" diff --git a/docs/contributing/developers_guide/dev_cellpy_data_structure.md b/docs/contributing/developers_guide/dev_cellpy_data_structure.md index 192d2193..fe9e905f 100644 --- a/docs/contributing/developers_guide/dev_cellpy_data_structure.md +++ b/docs/contributing/developers_guide/dev_cellpy_data_structure.md @@ -63,4 +63,7 @@ The `Data` object contains the data and the meta-data for the cell characterisat ## The cellpy file format -As default, cellpy stores files in the hdf5 format. +As default (cellpy 2.x / file version 9), cellpy stores files as a zip of +parquet tables plus ``meta.json`` (``.cellpy``). Older HDF5 layouts (v4–v8) +remain readable; pass a ``.h5`` path or ``cellpy_file_format="hdf5"`` to write +HDF5. See {doc}`../../getting_started/migration_v1_to_v2`. diff --git a/docs/getting_started/index.md b/docs/getting_started/index.md index 0e0e36ec..3e47b06e 100644 --- a/docs/getting_started/index.md +++ b/docs/getting_started/index.md @@ -7,4 +7,5 @@ installation.md configuration.md checkup.md basic_usage.md +migration_v1_to_v2.md ``` diff --git a/docs/getting_started/migration_v1_to_v2.md b/docs/getting_started/migration_v1_to_v2.md new file mode 100644 index 00000000..bf31e904 --- /dev/null +++ b/docs/getting_started/migration_v1_to_v2.md @@ -0,0 +1,84 @@ +# Migrating from cellpy 1.x to 2.x + +This guide covers the user-visible breaks between the 1.x line and cellpy 2.0: +on-disk files, the per-test metadata model, campaign merges, and selected API +cleanups. Keep the `v1.x` branch / 1.x PyPI releases if you need the old +behaviour unchanged until July 2027 (see the project maintenance note). + +## File format + +| | 1.x / v8 | 2.x / v9 | +|---|----------|----------| +| Default `save()` | HDF5 (`.h5` / `.hdf5` / `.cellpy` as HDF5) | **zip-of-parquet** + `meta.json` (`.cellpy`) | +| Escape hatch | — | `save("out.h5")` or `cellpy_file_format="hdf5"` / `"v8"` | +| What `load()` reads | v4–v8 HDF5 | **v4–v8 HDF5 and v9** (sniffs zip vs HDF5) | + +v9 stores raw / steps / summary (and optional fid) as parquet members inside a +zip, with typed metadata in `meta.json`. Parquet columns use **native** header +names plus a schema version stamp. + +### Convert once + +Old files keep working under `load()`. To rewrite a library to v9: + +```python +from cellpy import cellreader + +cell = cellreader.CellpyCell() +cell.load("legacy_run.h5") # v4–v8 HDF5 +cell.save("legacy_run.cellpy") # v9 zip-of-parquet +``` + +After conversion, prefer `.cellpy` for new writes. Keep a `.h5` copy only if a +downstream tool still requires HDF5. + +## Metadata model + +2.x exposes `Data.tests` — a `TestMetaCollection` keyed by compact `test_id` +(0 for a single unmerged test). The legacy boxes (`meta_common` / +`meta_test_dependent`) remain the in-memory source of truth for the active +test; extra tests live in the collection. + +- **v8 HDF5** still persists only the **active** test (legacy limitation). +- **v9 `.cellpy`** persists the **full** collection (active + extras), plus + units/limits and load provenance where available. + +Standalone helpers (cellpy-owned; `cellpycore.metadata.io` archive stubs stay +stubs): + +```python +from cellpy.readers.cellpy_file.meta_archive import ( + save_meta_archive, + load_meta_archive, +) +``` + +## Campaign merge + +`CellpyCell.merge(..., mode="campaign")` folds different tests into one object +with distinct `test_id` values on raw (and on steps after `make_step_table`). +Saving that object as **v9** round-trips both `TestMeta` rows and the +`test_id` columns. Saving as **v8** still drops non-active test metadata. + +```python +left.merge(right) # campaign (default for distinct tests) +left.save("campaign.cellpy") # keeps tests [0, 1] + test_id columns +``` + +## API cleanups worth knowing (#509) + +- Prefer `cellpy.get` as the entry point; `merge_cells` / `print_instruments` + are exported at package level. +- `make_summary(exclude_step_types=[...])` replaces the old no-effect selector + exclusion kwargs (those still warn only). +- Curve helpers live in `cellpy.readers.capacity_curves` with thin + `CellpyCell` delegates — call sites on `CellpyCell` stay the same. +- See [`DEPRECATIONS.md`](../../DEPRECATIONS.md) for the registered removal + schedule (`easyplot` → 2.0, etc.). + +## Dependencies + +cellpy 2.x depends on **`cellpycore`**. Release builds pin an exact +`cellpycore==X.Y.Z` so a given cellpy tag maps to one core revision. If you +develop against an editable sibling checkout, follow +`CONTRIBUTING.md` / the dual-repo sync notes — do not commit path overrides. diff --git a/pyproject.toml b/pyproject.toml index 205445b4..986daedc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -62,7 +62,7 @@ dependencies = [ # a known core revision. For dual-repo development, install the sibling # checkout editable after sync (see CONTRIBUTING.md); do not commit a # [tool.uv.sources] path override — it breaks Dependabot lock updates. - "cellpycore==0.2.1", + "cellpycore==0.2.2", "pydantic-settings>=2.14.2", "platformdirs>=4.10.0", ] diff --git a/tests/data/goldens/pipeline_smoke/summary.parquet b/tests/data/goldens/pipeline_smoke/summary.parquet index c9f7d978..6af53c3b 100644 Binary files a/tests/data/goldens/pipeline_smoke/summary.parquet and b/tests/data/goldens/pipeline_smoke/summary.parquet differ diff --git a/uv.lock b/uv.lock index 5e9d2bf3..2d6fb850 100644 --- a/uv.lock +++ b/uv.lock @@ -426,7 +426,7 @@ dev = [ [package.metadata] requires-dist = [ - { name = "cellpycore", specifier = "==0.2.1" }, + { name = "cellpycore", specifier = "==0.2.2" }, { name = "charset-normalizer" }, { name = "click" }, { name = "cookiecutter" }, @@ -493,16 +493,16 @@ dev = [ [[package]] name = "cellpycore" -version = "0.2.1" +version = "0.2.2" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "pandas" }, { name = "polars" }, { name = "pyarrow" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/2a/25/b225b9746954d626803dd2790487316639aced0334dcc5e7fd6c43121a7b/cellpycore-0.2.1.tar.gz", hash = "sha256:2624033e47fdc51bc9fa8d644ba6bf7ca8d69ee66f88ffbd922a819f5504843a", size = 837241, upload-time = "2026-07-15T18:16:59.823Z" } +sdist = { url = "https://files.pythonhosted.org/packages/28/5c/49aea4804168f05e5908a1dcf33b04a05a172b13554a1b7b6fef3fdb9cff/cellpycore-0.2.2.tar.gz", hash = "sha256:c76ec9a8da6a335d000cc654eed0acdcbc4e42012b903d102798bffceab2f8b1", size = 871522, upload-time = "2026-07-17T12:12:59.093Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/79/01/e405d5d9a365c68c761887263927f4c9ea97e1e70a826cfcaf5c1f096cb3/cellpycore-0.2.1-py3-none-any.whl", hash = "sha256:343d3a7eb6fb7b096e044a6eca1d74e073ac429bd2d85fd005f0871fa455c123", size = 90045, upload-time = "2026-07-15T18:16:58.504Z" }, + { url = "https://files.pythonhosted.org/packages/1b/c5/e8dee8412ef27357aad35e96d3c10f94b9a9628b171e2f76f45f889878ef/cellpycore-0.2.2-py3-none-any.whl", hash = "sha256:03736497da66ab3f34a106dea8ff491d3139cad1e0a3813efe593f551971d031", size = 90744, upload-time = "2026-07-17T12:12:57.694Z" }, ] [[package]]