Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate from Poetry to uv #578

Merged
merged 55 commits into from
Jan 13, 2025
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
8391643
Migrate pyproject from Poetry to uv
disrupted Jan 7, 2025
3b1f053
Configure package
disrupted Jan 7, 2025
86c1076
Fix and simplify just serve-docs
disrupted Jan 7, 2025
f8e2631
Update docs
disrupted Jan 7, 2025
e62e09c
Update CI
disrupted Jan 7, 2025
ef6154a
Temporarily disable custom hooks
disrupted Jan 7, 2025
ac1ca11
Apply Ruff fixes
disrupted Jan 7, 2025
edaee3c
Fix custom hooks
disrupted Jan 7, 2025
fa70335
Update schemas
disrupted Jan 7, 2025
0e1bbef
Update CLI docs
disrupted Jan 7, 2025
05ba82d
Fix generate docs env vars
disrupted Jan 7, 2025
0b45909
Separate docs job
disrupted Jan 7, 2025
5240fda
Configure pytest-asyncio to silence deprecation warning
disrupted Jan 7, 2025
557bc38
Fix dotenv test
disrupted Jan 7, 2025
3d31e1c
Fix test assertions using Pydantic type AnyUrl -> AnyHttpUrl
disrupted Jan 7, 2025
42c3a08
Refactor test to get rid of Pyright ignore pragmas
disrupted Jan 7, 2025
4ab0791
Fix test
disrupted Jan 7, 2025
7041467
Move custom hooks to docs job
disrupted Jan 7, 2025
5ec1028
Remove uv sync step from docs job
disrupted Jan 7, 2025
10c4f6f
Fix uv command
disrupted Jan 7, 2025
581513c
Fix package build & custom hooks
disrupted Jan 7, 2025
a1e23b7
Add publish snapshot job
disrupted Jan 7, 2025
5927d7b
Retry publish snapshot
disrupted Jan 7, 2025
d586d9e
Rename job
disrupted Jan 7, 2025
a5493d0
Set license via classifier
disrupted Jan 7, 2025
2060f68
Fix setuptools bug using workaround
disrupted Jan 7, 2025
e8e7aef
Set fixed tag for uv
disrupted Jan 8, 2025
39d4ede
Bump version and set snapshot time before publishing
disrupted Jan 8, 2025
a796e2e
Fix shellcheck warning
disrupted Jan 8, 2025
6965e9a
Fix action ref
disrupted Jan 8, 2025
65e553b
Fix step output
disrupted Jan 8, 2025
b6cc560
Fix output ref
disrupted Jan 8, 2025
c0a31b9
Fix unclean Git working directory for consecutive bumpversion calls
disrupted Jan 8, 2025
34a0828
Fix duplicate -dev suffix
disrupted Jan 8, 2025
2363aa1
Remove todo
disrupted Jan 8, 2025
d845e5c
Use generic bump-version-release workflow
disrupted Jan 8, 2025
02aa5bf
Publish to PyPI using uv
disrupted Jan 8, 2025
b9c1ece
Skip Windows jobs on draft PRs because they are slow
disrupted Jan 8, 2025
4e4ed58
Update actionlint
disrupted Jan 8, 2025
75c6803
Skip Windows jobs on draft PRs because they are slow
disrupted Jan 8, 2025
1e94f1e
Update runner to Ubuntu 24.04
disrupted Jan 8, 2025
0fecc0f
Try simplify condition
disrupted Jan 8, 2025
e5e24c9
Inspect github context
disrupted Jan 8, 2025
16dd377
Remove debug context
disrupted Jan 8, 2025
8bec02e
Move `kpops schema` command down
disrupted Jan 8, 2025
c60895d
Move format to separate job
disrupted Jan 8, 2025
b76f0fe
Remove changelog-config
disrupted Jan 8, 2025
405113c
Restore ruff format
disrupted Jan 8, 2025
f4bed07
Add comment about git restore
disrupted Jan 8, 2025
a14ef38
Configure cliff changelog
disrupted Jan 9, 2025
d38a082
Remove `@pytest.mark.asyncio` decorator
disrupted Jan 9, 2025
8a8f9e2
Merge remote-tracking branch 'origin/main' into uv
disrupted Jan 13, 2025
7810ef5
Refactor CI condition
disrupted Jan 13, 2025
44a1760
Refactor snapshot time & add todo
disrupted Jan 13, 2025
e3a2e8a
Fix missing extra dependencies
disrupted Jan 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[bumpversion]
current_version = 9.1.0
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>dev)(?P<timestamp>\d+)?)?
serialize =
{major}.{minor}.{patch}-{release}{timestamp}
{major}.{minor}.{patch}-{release}
{major}.{minor}.{patch}

[bumpversion:part:release]
first_value = dev
optional_value = release
values =
dev
release

[bumpversion:file:pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"

[bumpversion:file:kpops/const/__init__.py]
15 changes: 5 additions & 10 deletions .github/actions/update-docs/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,11 @@ inputs:
runs:
using: "composite"
steps:
- name: Install Python and set up Poetry
uses: bakdata/ci-templates/actions/python-setup-[email protected]
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.11"
poetry-version: "1.5.1"

- name: Install docs dependencies
shell: bash
run: |
poetry install --with docs
version: "0.5.14"

- name: Update gh-pages branch
shell: bash
Expand All @@ -47,12 +42,12 @@ runs:
shell: bash
if: ${{ inputs.release == 'false' }}
run: |
poetry run mike deploy ${{ inputs.version }} --push --config-file ./docs/mkdocs.yml
uv run --frozen --group=docs mike deploy ${{ inputs.version }} --push --config-file ./docs/mkdocs.yml
JakobEdding marked this conversation as resolved.
Show resolved Hide resolved

- name: Deploy ${{ inputs.version }} version (latest) of the documentation with mike
shell: bash
if: ${{ inputs.release == 'true' }}
run: |
sem_version=${{ inputs.version }}
major_minor_version=${sem_version%.*}
poetry run mike deploy "$major_minor_version" latest --update-aliases --push --config-file ./docs/mkdocs.yml
uv run --frozen --group=docs mike deploy "$major_minor_version" latest --update-aliases --push --config-file ./docs/mkdocs.yml
16 changes: 0 additions & 16 deletions .github/changelog-config.json

This file was deleted.

Empty file removed .github/scripts/__init__.py
Empty file.
5 changes: 0 additions & 5 deletions .github/scripts/get-dev-version.py

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Lint CI
name: Lint GitHub CI

on:
pull_request:
Expand All @@ -11,9 +11,9 @@ on:
jobs:
actionlint:
name: actionlint
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- name: Lint actions and workflows
uses: bakdata/ci-templates/actions/[email protected]
with:
action-lint-version: "v1.6.25"
action-lint-version: "v1.7.6"
2 changes: 1 addition & 1 deletion .github/workflows/add-issue-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
jobs:
add-issue-to-project:
name: Add issue to project board
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- name: Add the issue to the project
uses: actions/[email protected]
Expand Down
193 changes: 143 additions & 50 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,96 +12,188 @@ defaults:
run:
shell: bash

env:
UV_VERSION: "0.5.14"

jobs:
test:
name: Test
lint:
name: Lint
strategy:
fail-fast: false
matrix:
os:
- ubuntu-22.04
- ubuntu-24.04
- windows-2022
python-version: ["3.11", "3.12"]
is-draft-pr:
- ${{ github.event.pull_request.draft }}
include:
- python-version: "3.11"
os: ubuntu-24.04
format-for-github: true
exclude:
# skip Windows jobs on draft PRs because they are slow
- os: windows-2022
is-draft-pr: true
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v4
with:
submodules: true

- name: Install Python and set up Poetry
uses: bakdata/ci-templates/actions/python-setup-[email protected]
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.python-version }}
poetry-version: "1.7.1"

- name: Check Poetry lock file consistency
run: poetry lock --check
version: ${{ env.UV_VERSION }}

- name: Install dependencies
run: poetry install --no-interaction
- name: Check uv.lock
run: uv lock --check

- name: Lint (ruff)
run: |
if [[ "$RUNNER_OS" == "Linux" && "${{ matrix.python-version }}" == "3.11" ]]
if [[ "${{ matrix.format-for-github }}" == "true" ]]
then
poetry run ruff check . --config pyproject.toml --output-format=github --no-fix
uv run --frozen ruff check . --config pyproject.toml --output-format=github --no-fix
else
poetry run pre-commit run ruff-lint --all-files --show-diff-on-failure
uv run --frozen ruff check . --config pyproject.toml --no-fix
fi;
disrupted marked this conversation as resolved.
Show resolved Hide resolved

- name: Formatting (ruff)
run: poetry run pre-commit run ruff-format --all-files --show-diff-on-failure

- name: Typing (pyright)
run: |
if [[ "$RUNNER_OS" == "Linux" && "${{ matrix.python-version }}" == "3.11" ]]
if [[ "${{ matrix.format-for-github }}" == "true" ]]
then
echo "::add-matcher::.github/pyright-matcher.json"
poetry run pre-commit run pyright --all-files
else
poetry run pre-commit run pyright --all-files
fi;
uv run --frozen pyright
JakobEdding marked this conversation as resolved.
Show resolved Hide resolved

- name: Generate schema (kpops schema)
run: poetry run pre-commit run gen-schema --all-files --show-diff-on-failure

- name: Generate CLI Usage docs (typer-cli)
run: poetry run pre-commit run gen-docs-cli --all-files --show-diff-on-failure
format:
name: Format
runs-on: [ubuntu-24.04]
steps:
- uses: actions/checkout@v4

- name: Generate Environment variable docs
run: poetry run pre-commit run gen-docs-env-vars --all-files --show-diff-on-failure
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.11"
version: ${{ env.UV_VERSION }}

- name: Generate pipeline definitions
run: poetry run pre-commit run gen-docs-components --all-files --show-diff-on-failure
- name: ruff
run: uv run --frozen ruff format --check

- name: dprint
uses: dprint/[email protected]

test:
name: Test
strategy:
fail-fast: false
matrix:
os:
- ubuntu-24.04
- windows-2022
python-version: ["3.11", "3.12"]
is-draft-pr:
- ${{ github.event.pull_request.draft }}
exclude:
# skip Windows jobs on draft PRs because they are slow
- os: windows-2022
is-draft-pr: true
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v4
with:
submodules: true

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.python-version }}
version: ${{ env.UV_VERSION }}

- name: Test
run: poetry run pytest tests
run: uv run --frozen pytest tests

- name: Install docs dependencies
run: poetry install --with docs
docs:
name: Docs
runs-on: [ubuntu-24.04]

- name: Check markdown, toml, css formatting
uses: dprint/[email protected]
if: runner.os == 'Linux'
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.11"
version: ${{ env.UV_VERSION }}

- name: Generate schema (kpops schema)
run: uv run --frozen pre-commit run gen-schema --all-files --show-diff-on-failure

- name: Generate CLI Usage docs (typer-cli)
run: uv run --frozen pre-commit run gen-docs-cli --all-files --show-diff-on-failure

- name: Generate Environment variable docs
run: uv run --frozen pre-commit run gen-docs-env-vars --all-files --show-diff-on-failure

- name: Generate pipeline definitions
run: uv run --frozen pre-commit run gen-docs-components --all-files --show-diff-on-failure

- name: Test docs build (mkdocs)
run: poetry run mkdocs build -f docs/mkdocs.yml
run: uv run --frozen --group=docs mkdocs build -f docs/mkdocs.yml

publish-snapshot-version:
name: Publish snapshot to TestPyPI
needs: [test]
uses: bakdata/ci-templates/.github/workflows/[email protected]
with:
python-version: "3.11"
poetry-version: "1.7.1"
secrets:
pypi-token: ${{ secrets.TEST_PYPI_TOKEN }}
needs: [lint, format, test, docs]
disrupted marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.11"
version: ${{ env.UV_VERSION }}

- name: Generate snapshot time
id: snapshot-time
run: |
snapshot_time=$(python -c "from datetime import datetime; time = datetime.now(); print(time.strftime('%Y%m%d%H%M%S'))")
echo "snapshot-time=${snapshot_time}" >> "$GITHUB_OUTPUT"
disrupted marked this conversation as resolved.
Show resolved Hide resolved

- name: Bump version patch
id: bump-version
uses: bakdata/ci-templates/actions/[email protected]
with:
release-type: patch

- name: Restore clean Git working directory
# clean Git working directory is needed for the next bumpversion call
# we only need the version result from the first `patch` bump which is stored as a step output
run: git restore .

- name: Bump version snapshot time
uses: bakdata/ci-templates/actions/[email protected]
with:
release-type: patch
new-version: ${{ steps.bump-version.outputs.release-version }}${{ steps.snapshot-time.outputs.snapshot-time }} # e.g. 9.1.1-dev20250108102827

- name: Build
run: uv build

- name: Publish TestPyPI
run: uv publish
env:
UV_PUBLISH_URL: https://test.pypi.org/legacy/
UV_PUBLISH_TOKEN: ${{ secrets.TEST_PYPI_TOKEN }}

publish-docs-from-main:
runs-on: ubuntu-22.04
name: Publish docs (main)
runs-on: ubuntu-24.04
if: ${{ github.ref == 'refs/heads/main' }}
needs: [test]
needs: [lint, format, test, docs]
steps:
- uses: actions/checkout@v4

Expand All @@ -114,9 +206,10 @@ jobs:
version: main

publish-dev-docs-from-pr:
runs-on: ubuntu-22.04
name: Publish docs (dev)
runs-on: ubuntu-24.04
if: ${{ github.event_name == 'pull_request' }}
needs: [test]
needs: [lint, format, test, docs]
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -125,7 +218,7 @@ jobs:
# Checks to see if any files in the PR match one of the listed file types.
# This will return true if there's a file in docs folder that was added, deleted, or modified in the PR.
- name: Check if files in docs folder have changed
uses: dorny/paths-filter@v2
uses: dorny/paths-filter@v3
id: docs-changes
with:
filters: |
Expand Down
25 changes: 18 additions & 7 deletions .github/workflows/publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,21 @@ on:

jobs:
publish:
uses: bakdata/ci-templates/.github/workflows/[email protected]
with:
publish-to-test: false
python-version: "3.11"
poetry-version: "1.7.1"
secrets:
pypi-token: "${{ secrets.PYPI_TOKEN }}"
name: Publish to PyPI
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.11"
version: "0.5.14"
JakobEdding marked this conversation as resolved.
Show resolved Hide resolved

- name: Build
run: uv build

- name: Publish
run: uv publish
env:
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN }}
Loading
Loading