Write up major part of the changelog

openPMD · Jul 11, 2024 · a7d3d75 · a7d3d75
1 parent b1547eb
commit a7d3d75
Showing 1 changed file with 117 additions and 107 deletions.
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -18,78 +18,90 @@ Features
 """"""""
 
 - pybind11: require version 2.12.0+ #1220 #1322 #1637 #1489 #1640
-* 733ef8d35 Python: 3.8+ (#1502)
+- Python: require version 3.8+ #1502
 - ADIOS2:
-  * 58028ea0e ADIOS2 schema 2022_07_26, based on ADIOS2 modifiable attributes (#1310)
-  * 5fec41525 Joined arrays in ADIOS2 (#1382)
-  * 489038843 Remove ADIOS1 - Long Live ADIOS2 (#1419)
-    * 599a1cb4d Remove `FindADIOS.cmake` (#1560)
-  * e5f5ac49a added async write flag for ADIOS2 (#1460)
-  * 4de431c46 ADIOS2: Optionally write attributes only from given ranks (#1542)
-  * e668e8629 Adios2 warn groupbased encoding (#1498)
+
+  - Introduce group tables for more stable support of ADIOS2 steps #1310
+    Used in conjunction with modifiable attributes in ADIOS2 v2.9
+  - Support for "joined array" variable shape #1382
+  - Support Async write flag #1460
+  - Performance optimization for extreme-scale parallelism:
+    Optionally write attributes only from specified ranks #1542
+  - Group-Based encoding will now print warnings due to  #1498
+  - Remove ADIOS1 - Long Live ADIOS2 #1419 #1560
 - HDF5:
-  * 8faadde42 Initial support for HDF5 subfiling (#1580)
-  * 30e5bde87 HDF5: Explicit control over chunking (#1591)
-  * e7d7a4b2e HDF5IOHandler: Support for float128 on ARM64/PPC64 (#1364)
-  * 3bc460d15 HDF5: Handle unknown datatypes in datasets (#1469)
+
+  - Support for HDF5 subfiling #1580
+  - Explicit control over chunking #1591 #1600
+  - Support for float128 on ARM64/PPC64 #1364
+  - Better handling of unknown datatypes in datasets #1469
 - JSON/TOML:
-  * d64dbc29b Parallel JSON (#1475)
-  * 9ec90b68b TOML Backend (#1436)
-* 2e89f87ab Remove necessity for RecordComponent::SCALAR (#1154)
-  * 87cdc965b Fix Python container and pickle API (#1627)
-* 0387b6f7e Optional debugging output for AbstractIOHandlerImpl::flush() (#1495)
-  * 8e8d88e4b Update for OPENPMD_VERBOSE logging (#1574)
-  * 9184c2e8b Fix wrong logging of TOUCH task (#1643)
-* 9365031cc Better handling for file extensions (#1473)
-* c2e0e83f0 Filename extensions: Allow specifying wildcards (#1584)
-* 0baf09f5f Add rank table for locality-aware streaming (#1505)
+
+  - Parallel JSON (mostly for debugging purposes, separate output written per rank) #1475
+  - TOML Backend (by converting the in-/output of the JSON backend) #1436
+- API simplification: ``RecordComponent::SCALAR`` no longer necessary #1154 #1627
+- Allow specifying wildcards for filename extensions: ``simData_%T.%E`` #1584
+- Streaming: Add rank table for locality-aware streaming #1505
+- Optional debugging output for ``AbstractIOHandlerImpl::flush()`` via ``export OPENPMD_VERBOSE=1`` #1495 #1574 #1643
 - Miscellaneous API additions:
-  * b131cd2c4 Add visit-like pattern for RecordComponent (#1544)
-    * a3fe9b75d Remove constexpr from RecordComponent::visit() (#1582)
-  * 283b2fe35 Derive PatchComponent from RecordComponent (#1594)
+
+  - Add visit-like pattern for RecordComponent #1544 #1582
+  - Derive PatchRecordComponent from RecordComponent to give access to the full load/store API #1594
+  - More consistent handling for file extensions #1473
 
 Bug Fixes
 """""""""
 
 - ADIOS2
-  * a3c19888d ADIOS2: Ensure that a step is always active at write time (#1492)
-  * c124f8fde Warn on BP5+Blosc in ADIOS2 v2.9 up to patch level 1 (#1497)
-  * 95cbcb6da Fix CI issues for ADIOS2 v2.10: Add openPMD_HAVE_ADIOS2_BP5, datatype fixes in Python bindings (#1618)
+
+  - Ensure that a step is always active at write time #1492
+    Necessary for the BP5 engine
+  - Warning for BP5+Blosc in ADIOS2 v2.9 up to patch level 1 #1497
+    Unreadable datasets might silently be created due to a bug in ADIOS2 v2.9.0 and v2.9.1
+    https://github.com/ornladios/ADIOS2/issues/3504
+  - Some adjustments for ADIOS2 v2.10 #1618
+    This adds the ``openPMD_HAVE_ADIOS2_BP5`` macro and introduces some datatype fixes in Python bindings
 - HDF5
-  * ef8b9998d HDF5: Fix Char Type Matching (#1433)
-  * c8c960afc Fix hdf5 dataset specific config (#1600)
+
+  - Fix Char Type Matching #1433
 - CMake:
-  * d0a0e1d0d openpmd-pipe: set correct install permissions (#1459)
-  * e01f72176 Fix CMake: HDF5 Libs are PUBLIC (#1520)
+
+  - Set correct install permissions for ``openpmd-pipe`` #1459
+  - HDF5 Libraries are ``PUBLIC`` #1520
 - Warnings:
-  * e2a79ae6c Fix gcc9 warning (#1429)
-  * 15feda471 ADIOS 2.9: Avoid Unused Param Warning (#1503)
+
+  - Fix gcc9 warning #1429
+  - ADIOS2 v2.9: Avoid Unused Param Warning #1503
 - Python:
-  * acf1c11d6 Python: Fix ODR Violation (#1521)
-  * 996ef8419 Python: Fix Init Order (#1547)
-  * 3a8a22053 Fix: Strings with a single char in Python (#1585)
+
+  - ODR Violation #1521
+  - Init Order #1547
+  - Strings with a single char #1585
+  * Fixes and documentation for Pybind v2.12.0 and Numpy 2.0 #1637
 - Tooling:
-  * 6b4c455ac openpmd-pipe: fix handling of constant components (#1530)
+
+  - ``openpmd-pipe``: fix handling of constant components #1530
 - Performance:
-  * e4ce81f92 Fix dirtyRecursive() performance for Series with many steps (#1598)
-* 89d3a2b88 Bug fix: Linear read mode cannot directly access specific file of file-based Series (#1533)
-* 413b1c540 RecordComponent: Properly handle uninitialized datasets (#1316)
-* c09d9bc9f 5e89ef9a9 Don't require unitSI when reading patch record component (#1470 #1482)
-* c25b31756 Fixes for variable-based encoding in backends without step support (#1484)
-* 442481aaf Fix `unique_ptr<T, Del>` constructor of UniquePtrWithLambda (#1552)
-* 609fcdd93 Replace openPMD_Datatypes global with function (#1509)
-* 9ad33b07a Fix Attribute copy/move constructors (#1545)
-* 2e0b016bc Fix double mesh.read() call (#1535)
-* c34bcea57 Partially revert #1368 (#1573)
-* d5524ec16 Fix availableChunks for READ_LINEAR in ADIOS2 (#1586)
-* d9fce66dd Don't allow Container insertion in READ_LINEAR (#1590)
-* 045e416ab SerialIOTest: Clang-Tidy Fixes (#1599)
-* 5940f7cac Read JSON config in parallel (#1605)
-* 8c5728516 Span API: Set dirty flag (#1615)
-* 5d9fb34c2 Workaround for independent writes to Iterations in parallel, better detection of BP5 which in turn uncovers more instances of the first issue (#1619)
-* 534706f88 Fix particle patches flush api (#1626)
-  * 7f5c33d98 Fix: Add back iteration.close() call (#1641)
-* a2071184c Regexes: Sanitize user input (#1624)
+
+  - Fix ``dirtyRecursive()`` performance for Series with many steps #1598 #1615
+  - Fix flushing performance for file-based Series with many steps #1642
+- Workaround for independent writes to Iterations in parallel #1619
+  This includes better detection of BP5 which in turn uncovers more instances of the first issue
+- Regexes: Sanitize user input to avoid Regex injection #1624
+- Fix particle patches flush api #1626 #1641
+- RecordComponent: Properly handle uninitialized datasets in ``RecordComponent`` #1316
+- Don't require unitSI when reading patch record component #1470 #1482
+- Linear read mode was not able to directly access specific file of file-based Series #1533
+- Fixes for variable-based encoding in backends without step support #1484
+- Fix availableChunks for ``READ_LINEAR`` in ADIOS2 #1586
+- Read JSON config in parallel #1605
+- Partially revert #1368 to re-enable a warning #1573
+- Fix ``unique_ptr<T, Del>`` constructor of ``UniquePtrWithLambda`` #1552
+- SerialIOTest: Clang-Tidy Fixes #1599
+- Replace ``openPMD_Datatypes`` global with function #1509
+- Fix ``Attribute`` copy/move constructors #1545
+- Fix duplicate ``mesh.read()`` call #1535
+- Disallow Container insertion in ``READ_LINEAR`` #1590
 
 
 Breaking Changes
@@ -102,61 +114,59 @@ Other
 
 - Catch2: updated to 2.13.10 #1299 #1344
 - Tests & Examples:
-  * 77c73e208 Fix deprecated storeChunk APIs in first read/write examples (#1435)
-  * 6bb7d8205 Streaming examples: Set WAN as default transport (#1511)
-  * 9d5bf968f replace extent in weighting and displacement (#1510)
+
+  - Rewrite deprecated storeChunk APIs in first read/write examples #1435
+  - Streaming examples: Set WAN as default transport #1511
+  - Fix records length in "9_particle_write_serial.py" #1510
 - CI
-  * 6940f82c5 GitHub Actions: macOS has 3 Cores (#1421)
-  * a8b39b918 CI: macOS-11 Update (#1446)
-  * 8b35d09cb CI: macOS 11.0+ (#1486)
-  * b57ad0389 CI: Doxygen 1.9.7 Broken (#1464)
-  * bb808277a CI: oneAPI 2023.2.0 (#1478)
-  * 8a7ec6173 CI fix: type comparison in Python (#1490)
-  * 1cf46fd90 CI: Old CTest (#1519)
-  * 2384af017 Fix `chmod` in `download_samples.sh` (#1518)
-  * 71aa0e98d Add all the performance-* clang-tidy checks (#1532)
-  * 6b963badb Workaround for current CI issues with clang14 on MacOS (#1565)
-    * 51caab374 Seems we need the MPI workaround for Conda now too..... (#1628)
-  * 55af0dbd2 CI: Linux aarch64/arm64 (#1517)
-  * d24c5d26b CI: Fix CircleCI "six" (#1596)
-  * 3daaa1409 CI fixes and documentation for Pybind v2.12.0 and Numpy 2.0 (#1637)
+
+  - 55af0dbd2 Linux aarch64/arm64 #1517
+  - macOS 11.0+ #1446 #1486
+  - oneAPI 2023.2.0 #1478
+  - Update .readthedocs.yml #1438
+  - GitHub Actions: macOS has 3 Cores #1421
+  - Doxygen 1.9.7 Broken #1464
+  - Fix type comparison in Python #1490
+  - Adapt to removed CTest CLI #1519
+  - Fix ``chmod`` in ``download_samples.sh`` #1518
+  - Workaround for bugs in CLI of ``mpiexec`` #1565 #1628
+  - Fix CircleCI "six" #1596
 - CMake
-  * 3c063e8bb CMake: Warnings on AppleClang (#1496)
-  * e594d4716 CMake: Warn and Continue on Empty HDF5_VERSION (#1512)
-  * 72ab3d12d Update cmake minimum required (#1558)
+  - Update cmake minimum required to 3.5 for third-party dependencies ``nlohmann::json`` and ``toruniina::toml11`` #1558
+  - Warnings on AppleClang #1496
+  - Warn and Continue on Empty HDF5_VERSION #1512
 - Docs
-  * 335071a38 More careful documentation of streaming API (#1430)
-  * 28fd5c81d Doc: Fix Bib Authors (#1434)
-  * 87d019051 Doc: Citation More Authors (#1539)
-  * 991d0d9e7 Doc: HELPMI Support (#1555)
-  * 0c8eba527 Docs: Research Groups of Maintainers (#1566)
-  * bb98f66eb Post 0.15.0 Changelog Template (#1420)
-  * 55b8b96c5 Update .readthedocs.yml (#1438)
-  * 4423be428 Docs: Linking to C++ Projects (#1445)
-  * 7b33805d5 Remove schema 2021 from documentation (#1451)
-  * 033af1bdc Document OpenMPI-ROMIO/HDF5/Chunking issue (#1441)
-  * dfc0a31e3 Doc: HDF5_DO_MPI_FILE_SYNC (#1427)
-  * e53377640 Fix small documentation issues after 0.15 release (#1440)
-  * f6b23a59c Doc: Sphinx Copybutton and Design (#1461)
-  * 32aa2cb49 Docs: Analysis (#1444)
-  * 2ee8a1695 Sphinx: Limit <7.2 (#1541)
-  * ac316677f Doc: Python 3.12 (#1549)
-  * 7296948d1 Add documentation for typical use cases of openpmd-pipe (#1578)
-  * 7bb2948bb Fix docstring for MyPath (#1587)
-  * 3daaa1409 CI fixes and documentation for Pybind v2.12.0 and Numpy 2.0 (#1637)
+  - Document typical Analysis workflows #1444
+  - Add documentation for typical use cases of openpmd-pipe #1578
+  - Authors
+   This includes: Synchronizing library and standard authors #1434, citing more authors #1539, support by the HELPMI project #1555, citing the research groups of maintainers #1566
+  - Document how to link to C++ Projects #1445
+  - More careful documentation of streaming API #1430
+  - Post 0.15.0 Changelog Template #1420
+  - Document OpenMPI-ROMIO/HDF5/Chunking issue #1441
+  - Document ``HDF5_DO_MPI_FILE_SYNC`` #1427
+  - Remove schema 2021 from documentation #1451
+  - Fix small documentation issues after 0.15 release #1440
+  - Add Sphinx Copybutton and Design #1461
+  - Sphinx: Limit <7.2 #1541
+  - Document that we support Python 3.12 #1549
+  - Fix docstring for MyPath #1587
 - Python
-  * 3acf5659f Python bindings: Release GIL during IO wait operations (#1381)
-  * 9ab2ecd78 Add Python binding for myPath (#1463)
-  * 198c3f966 Update __repr__ method of major objects in openPMD hierarchy (#1476)
-  * 0e4bab2da Python 3.12: Remove Distutils (#1508)
+
+  - Python bindings: Release GIL during IO wait operations #1381
+  - Add Python binding for myPath #1463
+  - Update __repr__ method of major objects in openPMD hierarchy #1476
+  - Python 3.12: Remove Distutils #1508
 - Tooling
-  * f875c1066 Use lazy imports for dask and pandas (#1442)
-  * 358e0abf7 Pandas DataFrames: Add Row Column Name (#1501)
-  * 99daca78e make it possible to manually set chunks when loading dask arrays (#1477)
-* e9bfa32ef Better error message when loading to a buffer with mismatched type (#1452)
-* 8b087368f Unknown openPMD version in data: Add upgrade hint (#1528)
-* 17e757f9f Refactor: Extract ADIOS2 BufferedActions struct to own file, rename to ADIOS2File (#1577)
-* 805c7606a Print a hint on what might be wrong when retrieveSeries fails (#1610)
+
+  - Use lazy imports for dask and pandas #1442
+  - Pandas DataFrames: Add Row Column Name #1501
+  - Add API to manually set chunks when loading dask arrays #1477
+- Add all the performance-* clang-tidy checks #1532
+- Better error message when loading to a buffer with mismatched type #1452
+- Unknown openPMD version in data: Add upgrade hint #1528
+- Print a hint on what might be wrong when retrieveSeries fails #1610
+- Refactor: Extract ADIOS2 BufferedActions struct to own file, rename to ADIOS2File #1577
 
 0.15.1
 ------