Script: New Version (#1467)

* Script: New Version

Adding a script to update all version notes programmatically for
releases.

Add maintainer docs.

Co-authored-by: Franz Pöschel <[email protected]>

docs/source/index.rst

@@ -167,4 +167,5 @@ Maintenance
    :maxdepth: 1
    :hidden:
 
-   maintenance/release
+   maintenance/release_github
+   maintenance/release_channels

docs/source/maintenance/release.rst → docs/source/maintenance/release_channels.rst

@@ -1,8 +1,10 @@
-.. _maintenance-release:
+.. _maintenance-release-channels:
 
 Release Channels
 ================
 
+After tagging and releasing a new openPMD-api version on GitHub, we update package managers that ship openPMD-api.
+
 Spack
 -----
 
@@ -15,7 +17,7 @@ Example workflow for a new release:
 
 - https://github.com/spack/spack/pull/14018
 
-Please ping `@ax3l <github.com/ax3l>`_ in your pull-request.
+Please ping `@ax3l <github.com/ax3l>`__ in your pull-request.
 
 
 Conda-Forge

docs/source/maintenance/release_github.rst

@@ -0,0 +1,53 @@
+.. _maintenance-release-github:
+
+New Version on GitHub
+=====================
+
+These are the steps we perform to create a new openPMD-api release.
+
+Regular Release
+---------------
+
+As a first step, we merge in all pull requests and resolve all issued that we have assigned to the release's "Milestone" on GitHub.
+For example, here are all `issues and pull requests <https://github.com/openPMD/openPMD-api/milestone/8>`__ on the version ``0.16.0`` release.
+
+Then, we prepare a pull request that updates the ``CHANGELOG.rst`` and upgrade guide in ``NEWS.rst``, example PR: `Release notes: 0.16 <https://github.com/openPMD/openPMD-api/pull/1648>`__
+In the same PR, we bump up the version in CMake files and documentation using our ``new_version.py`` script.
+
+Once the PR is merged, we:
+
+#. Wait for the CI to finish: there will be one more auto-commit added to the ``dev`` branch, updating out ``.pyi`` stub files
+#. Pull the latest ``dev`` branch to our local machine
+#. Add a GPG-signed tag, e.g., ``git tag -s 0.16.0``: see old releases for the format of this tag, e.g., use ``git show 0.15.0``. The text is from the top of ``CHANGELOG.rst``.
+#. Upload the GPG-signed tag to mainline, e.g., ``git push -u mainline 0.16.0``
+#. Click `Draft a new release <https://github.com/openPMD/openPMD-api/releases>`__ on GitHub, select the newly created tag.
+   Fill the text fields using the same format that you see for `earlier releases <https://github.com/openPMD/openPMD-api/releases>`__, again, based on the top of the text in ``CHANGELOG.rst``.
+   Skip the DOI badge for this release step.
+#. If you don't have GPG properly set up for your git, then you can just do the last step, which then also creates a tag.
+   Be sure to use the same version scheme for the tag, i.e., we do *not* prefix our tags with ``v`` or something of that kind!
+#. Go to `Rodare <https://doi.org/10.14278/rodare.27>`__ and wait for the release to arrive.
+   If there are issues, contact Rodare/HZDR IT support and check under `Settings - Webhooks <https://github.com/openPMD/openPMD-api/settings/hooks>`__ if the release was delivered.
+#. Once the Rodare DOI is auto-created, click on the badge on the right hand side of the page, copy the Markdown code, and edit your `newly created release <https://github.com/openPMD/openPMD-api/releases>`__ text on GitHub to add the badge as you see in earlier releases.
+
+That's it!
+
+
+Backport (Bugfix) Release
+-------------------------
+
+For bugfix releases, we generally follow the same workflow as for regular releases.
+
+The main difference is that we:
+
+#. Start a new branch from the release we want to backport to.
+   We name the branch ``release-<version>``, e.g., backports on the ``0.15.0`` release for a ``0.15.1`` release are named ``release-0.15.1``.
+#. We add pull requests (usually we ``git cherry-pick`` commits) to that branch until we have all fixes collected.
+   Backport releases try to not add features and to not break APIs!
+#. Then, we follow the same workflow as above, but we tag on the ``release-<version>`` branch instead of the ``dev`` branch.
+#. Once we uploaded the new tag to mainline and created the GitHub release, we remove the ``release-<version>`` branch from our mainline repo.
+   The new tag we added contains all history we need if we wanted to do a follow-up bugfix, e.g., a ``release-0.15.2`` branch based on the ``0.15.1`` release.
+
+As general guidance, we usually only fix bugs on the *latest* regular release.
+Don't sweat it - if it is too hard to backport a fix, consider doing a timely new regular version release.
+
+That's it!

new_version.py

@@ -0,0 +1,129 @@
+#!/usr/bin/env python3
+#
+# Copyright 2021-2023 Axel Huebl
+#
+# This file is part of openPMD-api.
+#
+
+# This file is a maintainer tool to bump the versions inside openPMD-api's
+# source directory at all places where necessary.
+#
+from pathlib import Path
+import re
+import sys
+
+# Maintainer Inputs ###########################################################
+
+print(
+    """Hi there, this is an openPMD maintainer tool to update the source
+code of openPMD-api to a new version.
+For it to work, you need write access on the source directory and
+you should be working in a clean git branch without ongoing
+rebase/merge/conflict resolves and without unstaged changes."""
+)
+
+# check source dir
+# REPO_DIR = Path(__file__).parent.parent.parent.absolute()
+REPO_DIR = Path(__file__).parent.absolute()
+print(f"\nYour current source directory is: {REPO_DIR}")
+
+REPLY = input("Are you sure you want to continue? [Y/n] ")
+print()
+if REPLY not in ["Y", "y", ""]:
+    print("You did not confirm with 'y', aborting.")
+    sys.exit(1)
+
+MAJOR = input("MAJOR version? (e.g., 1) ")
+MINOR = input("MINOR version? (e.g., 0) ")
+PATCH = input("PATCH version? (e.g., 0) ")
+SUFFIX = input("SUFFIX? (e.g., dev) ")
+
+VERSION_STR = f"{MAJOR}.{MINOR}.{PATCH}"
+VERSION_STR_SUFFIX = VERSION_STR + (f"-{SUFFIX}" if SUFFIX else "")
+
+print()
+print(f"Your new version is: {VERSION_STR_SUFFIX}")
+
+# Recover the old version from the Readme.
+# Do not use CMakeLists.txt as it might already contain the upcoming version
+# code.
+with open(str(REPO_DIR.joinpath("README.md")), encoding="utf-8") as f:
+    for line in f:
+        match = re.search(r"find_package.*openPMD *([^ ]*) *CONFIG\).*", line)
+        if match:
+            OLD_VERSION_STR = match.group(1)
+            break
+
+print(f"The old version is: {OLD_VERSION_STR}")
+print()
+
+
+REPLY = input("Is this information correct? Will now start updating! [y/N] ")
+print()
+if REPLY not in ["Y", "y", ""]:
+    print("You did not confirm with 'y', aborting.")
+    sys.exit(1)
+
+
+# Ask for new #################################################################
+
+print("""We will now run a few sed commands on your source directory.\n""")
+
+# Updates #####################################################################
+
+# run_test.sh (used also for Azure Pipelines)
+cmakelists_path = str(REPO_DIR.joinpath("CMakeLists.txt"))
+with open(cmakelists_path, encoding="utf-8") as f:
+    cmakelists_content = f.read()
+    cmakelists_content = re.sub(
+        r"^(project.*openPMD.*VERSION *)(.*)(\).*)$",
+        r"\g<1>{}\g<3>".format(VERSION_STR),
+        cmakelists_content,
+        flags=re.MULTILINE,
+    )
+
+with open(cmakelists_path, "w", encoding="utf-8") as f:
+    f.write(cmakelists_content)
+
+
+def generic_replace(filename):
+    filename = str(REPO_DIR.joinpath(filename))
+    with open(filename, encoding="utf-8") as f:
+        content = f.read()
+        content = re.sub(re.escape(OLD_VERSION_STR), VERSION_STR, content)
+
+    with open(filename, "w", encoding="utf-8") as f:
+        f.write(content)
+
+
+for file in ["docs/source/dev/linking.rst", "README.md"]:
+    generic_replace(file)
+
+version_hpp_path = str(REPO_DIR.joinpath("include/openPMD/version.hpp"))
+with open(version_hpp_path, encoding="utf-8") as f:
+    version_hpp_content = f.read()
+
+    def replace(key, value):
+        global version_hpp_content
+        version_hpp_content = re.sub(
+            r"^(#define OPENPMDAPI_VERSION_{}) .*$".format(re.escape(key)),
+            r"\1 {}".format(value),
+            version_hpp_content,
+            flags=re.MULTILINE,
+        )
+
+    replace("MAJOR", MAJOR)
+    replace("MINOR", MINOR)
+    replace("PATCH", PATCH)
+    replace("LABEL", '"{}"'.format(SUFFIX))
+
+with open(version_hpp_path, "w", encoding="utf-8") as f:
+    f.write(version_hpp_content)
+
+# Epilogue ####################################################################
+
+print(
+    """Done. Please check your source, e.g. via
+  git diff
+now and commit the changes if no errors occurred."""
+)