From 6fea69a84448677110b92a2d6619bafda2094b96 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 13 Oct 2024 13:25:25 +0900 Subject: [PATCH 01/11] docs: rules_python bzlmod GA and 1.0 prep This is documenting the current state and closing the last remaining TODO items for 1.0 release. Work towards #1361. --- BZLMOD_SUPPORT.md | 39 ++++++++++++++++++++++++++------------- docs/index.md | 44 +++++++++++++++++++++++++++++++++----------- 2 files changed, 59 insertions(+), 24 deletions(-) diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index d3d0607511..2c1c5e4831 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -2,10 +2,12 @@ ## `rules_python` `bzlmod` support -- Status: Beta +- Status: GA - Full Feature Parity: No + - `rules_python`: Yes + - `rules_python_gazelle_plugin`: No (see below). -Some features are missing or broken, and the public APIs are not yet stable. +In general `bzlmod` has more features than `WORKSPACE` and users are encouraged to migrate. ## Configuration @@ -27,15 +29,6 @@ A user does not use `local_path_override` stanza and would define the version in A second example, in [examples/bzlmod_build_file_generation](examples/bzlmod_build_file_generation) demonstrates the use of `bzlmod` to configure `gazelle` support for `rules_python`. -## Feature parity - -This rule set does not have full feature partity with the older `WORKSPACE` type configuration: - -1. Gazelle does not support finding deps in sub-modules. For instance we can have a dep like ` "@our_other_module//other_module/pkg:lib",` in a `py_test` definition. -2. We have some features that are still not fully flushed out, and the user interface may change. - -Check ["issues"](/bazelbuild/rules_python/issues) for an up to date list. - ## Differences in behavior from WORKSPACE ### Default toolchain is not the local system Python @@ -52,10 +45,30 @@ platforms. If you want to use the same toolchain as what WORKSPACE used, then manually register the builtin Bazel Python toolchain by doing `register_toolchains("@bazel_tools//tools/python:autodetecting_toolchain")`. -**IMPORTANT: this should only be done in a root module, and may intefere with +**IMPORTANT: this should only be done in a root module, and may interfere with the toolchains rules_python registers**. NOTE: Regardless of your toolchain, due to [#691](https://github.com/bazelbuild/rules_python/issues/691), `rules_python` still relies on a local Python being available to bootstrap the program before -handing over execution to the toolchain Python. +handing over execution to the toolchain Python. To override this behaviour see +{obj}`--bootstrap_impl=script`, which switches to `bash`-based bootstrap on +UNIX systems. + +### Better PyPI package downloading on bzlmod + +On `bzlmod` users have the option to use the `bazel_downloader` to download packages +and work correctly when `host` platform is not the same as the `target` platform. This +provides faster package download times and integration with the credentials helper. + +### Extra targets in `whl_library` repos + +Due to how `bzlmod` is designed and the visibility rules that it enforces, it is best to use +the targets in the `whl` repos as they are and not rely on using the `annotations` API to +add extra targets to so-called `spoke` repos. For alternatives that should cover most of the +existing usecases please see: +* {bzl:obj}`py_console_script_binary` to create `entry_point` targets. +* {bzl:obj}`whl_filegroup` to extract filegroups from the `whl` targets (e.g. `@pip//numpy:whl`) +* {bzl:obj}`pip.override` to patch the downloaded `whl` files. Using that you + can change the `METADATA` of the `whl` file that will influence how + `rules_python` code generation behaves. diff --git a/docs/index.md b/docs/index.md index c06c31ed44..2d5dd99bfc 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,32 +1,54 @@ # Python Rules for Bazel -rules_python is the home of the core Python rules -- `py_library`, -`py_binary`, `py_test`, `py_proto_library`, and related symbols that provide the basis for Python -support in Bazel. It also contains package installation rules for integrating with PyPI and other indices. +`rules_python` is the home for 4 major components: +* the core Python rules -- `py_library`, `py_binary`, `py_test`, + `py_proto_library`, and related symbols that provide the basis for Python + support in Bazel. + + This is subject to our breaking change policy outlined in the . +* Package installation rules for integrating with PyPI and other SimpleAPI + complying indexes. + + This is still `experimental` and the APIs might change more often than the + core rules or you may experience regressions between the minor releases. In + that case, please raise tickets to the GH issues bug tracker. +* `sphinxdocs` rules allowing users to generate documentation from bazel or + Python source code. + + This is available as is and without any guarantees. The semantic versioning + used by `rules_python` does not apply to bazel rules or the output. +* `gazelle` plugin for generating `BUILD.bazel` files based on Python source + code. + + This is available as is and without any guarantees. The semantic versioning + used by `rules_python` does not apply to the plugin. Documentation for rules_python lives here and in the [Bazel Build Encyclopedia](https://docs.bazel.build/versions/master/be/python.html). Examples are in the {gh-path}`examples` directory. -Currently, the core rules build into the Bazel binary, and the symbols in this -repository are simple aliases. However, we are migrating the rules to Starlark and removing them from the Bazel binary. Therefore, the future-proof way to depend on Python rules is via this repository. See -{ref}`Migrating from the Bundled Rules` below. +When using bazel 6, the core rules built into the Bazel binary, and the symbols +in this repository are simple aliases. However, on bazel 7 and above starlark +implementation in this repository is used. +See {ref}`Migrating from the Bundled Rules` below. The core rules are stable. Their implementation in Bazel is subject to Bazel's [backward compatibility policy](https://docs.bazel.build/versions/master/backward-compatibility.html). Once migrated to rules_python, they may evolve at a different rate, but this repository will still follow [semantic versioning](https://semver.org). -The Bazel community maintains this repository. Neither Google nor the Bazel team provides support for the code. However, this repository is part of the test suite used to vet new Bazel releases. See -{gh-path}`How to contribute ` for information on our development workflow. +The Bazel community maintains this repository. Neither Google nor the Bazel +team provides support for the code. However, this repository is part of the +test suite used to vet new Bazel releases. See {gh-path}`How to contribute +` for information on our development workflow. ## Bzlmod support -- Status: Beta -- Full Feature Parity: No +- Status: GA -See {gh-path}`Bzlmod support ` for more details +See {gh-path}`Bzlmod support ` for any behaviour differences between +`bzlmod` and `WORKSPACE`. ## Migrating from the bundled rules From 746492f8a9b7553c468e607f56e9de94e20408ab Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 13 Oct 2024 22:26:29 +0900 Subject: [PATCH 02/11] Improve docs --- BZLMOD_SUPPORT.md | 7 ++-- docs/getting-started.md | 33 ++++++--------- docs/index.md | 91 ++++++++++++++++++++++++----------------- 3 files changed, 70 insertions(+), 61 deletions(-) diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index 2c1c5e4831..f7b7044a04 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -51,9 +51,10 @@ the toolchains rules_python registers**. NOTE: Regardless of your toolchain, due to [#691](https://github.com/bazelbuild/rules_python/issues/691), `rules_python` still relies on a local Python being available to bootstrap the program before -handing over execution to the toolchain Python. To override this behaviour see -{obj}`--bootstrap_impl=script`, which switches to `bash`-based bootstrap on -UNIX systems. +handing over execution to the toolchain Python. + +To override this behaviour see {obj}`--bootstrap_impl=script`, which switches +to `bash`-based bootstrap on UNIX systems. ### Better PyPI package downloading on bzlmod diff --git a/docs/getting-started.md b/docs/getting-started.md index 45d1962ad8..99caf33aa7 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,7 +1,7 @@ # Getting started This doc is a simplified guide to help get started started quickly. It provides -a simplified introduction to having a working Python program for both bzlmod +a simplified introduction to having a working Python program for both `bzlmod` and the older way of using `WORKSPACE`. It assumes you have a `requirements.txt` file with your PyPI dependencies. @@ -23,11 +23,11 @@ bazel_dep(name = "rules_python", version = "0.0.0") pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( - hub_name = "my_deps", + hub_name = "pypi", python_version = "3.11", requirements_lock = "//:requirements.txt", ) -use_repo(pip, "my_deps") +use_repo(pip, "pypi") ``` ## Using a WORKSPACE file @@ -38,19 +38,14 @@ using Bzlmod. Here is a simplified setup to download the prebuilt runtimes. ```starlark load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") - -# Update the SHA and VERSION to the lastest version available here: -# https://github.com/bazelbuild/rules_python/releases. - -SHA="84aec9e21cc56fbc7f1335035a71c850d1b9b5cc6ff497306f84cced9a769841" - -VERSION="0.23.1" +# Update the snippet based on the latest release below +# https://github.com/bazelbuild/rules_python/releases http_archive( name = "rules_python", - sha256 = SHA, - strip_prefix = "rules_python-{}".format(VERSION), - url = "https://github.com/bazelbuild/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION), + sha256 = "ca77768989a7f311186a29747e3e95c936a41dffac779aff6b443db22290d913", + strip_prefix = "rules_python-0.36.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/0.36.0/rules_python-0.36.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -66,14 +61,12 @@ python_register_toolchains( python_version = "3.11", ) -load("@python_3_11//:defs.bzl", "interpreter") - load("@rules_python//python:pip.bzl", "pip_parse") pip_parse( - ... - python_interpreter_target = interpreter, - ... + name = "pypi", + python_interpreter_target = "@python_3_11_host//:python", + requirements_lock = "//:requirements.txt", ) ``` @@ -89,8 +82,8 @@ py_binary( name = "main", srcs = ["main.py"], deps = [ - "@my_deps//foo", - "@my_deps//bar", + "@pypi//foo", + "@pypi//bar", ] ) ``` diff --git a/docs/index.md b/docs/index.md index 2d5dd99bfc..8482705570 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,61 +1,71 @@ # Python Rules for Bazel -`rules_python` is the home for 4 major components: -* the core Python rules -- `py_library`, `py_binary`, `py_test`, - `py_proto_library`, and related symbols that provide the basis for Python - support in Bazel. +`rules_python` is the home for 4 major components with varying maturity levels. - This is subject to our breaking change policy outlined in the . -* Package installation rules for integrating with PyPI and other SimpleAPI - complying indexes. +:::{topic} Core rules - This is still `experimental` and the APIs might change more often than the - core rules or you may experience regressions between the minor releases. In - that case, please raise tickets to the GH issues bug tracker. -* `sphinxdocs` rules allowing users to generate documentation from bazel or - Python source code. +The core Python rules -- `py_library`, `py_binary`, `py_test`, +`py_proto_library`, and related symbols that provide the basis for Python +support in Bazel. - This is available as is and without any guarantees. The semantic versioning - used by `rules_python` does not apply to bazel rules or the output. -* `gazelle` plugin for generating `BUILD.bazel` files based on Python source - code. +When using bazel 6, the core rules built into the Bazel binary, and the symbols +in this repository are simple aliases. On bazel 7 and above `rules_python` uses +a separate Starlark implementation, +see {ref}`Migrating from the Bundled Rules` below. - This is available as is and without any guarantees. The semantic versioning - used by `rules_python` does not apply to the plugin. +Once rules_python 1.0 is released, they will follow +[semantic versioning](https://semver.org) and the breaking change policy +outlined in the [support](support) page. -Documentation for rules_python lives here and in the -[Bazel Build Encyclopedia](https://docs.bazel.build/versions/master/be/python.html). +::: -Examples are in the {gh-path}`examples` directory. +:::{topic} PyPI integration -When using bazel 6, the core rules built into the Bazel binary, and the symbols -in this repository are simple aliases. However, on bazel 7 and above starlark -implementation in this repository is used. -See {ref}`Migrating from the Bundled Rules` below. +Package installation rules for integrating with PyPI and other SimpleAPI +complying indexes. + +These rules are still `experimental` and the APIs might change more often +than the core rules or you may experience regressions between the minor +releases. In that case, please raise tickets to the GH issues bug tracker. + +::: + +:::{topic} Sphinxdocs + +`sphinxdocs` rules allowing users to generate documentation from bazel or +Python source code. + +This is available as is and the semantic versioning used by `rules_python` does +not apply to bazel rules or the output. + +::: + +:::{topic} Gazelle plugin -The core rules are stable. Their implementation in Bazel is subject to Bazel's -[backward compatibility policy](https://docs.bazel.build/versions/master/backward-compatibility.html). -Once migrated to rules_python, they may evolve at a different -rate, but this repository will still follow [semantic versioning](https://semver.org). +`gazelle` plugin for generating `BUILD.bazel` files based on Python source +code. + +This is available as is and the semantic versioning used by `rules_python` does +not apply to the plugin. + +::: The Bazel community maintains this repository. Neither Google nor the Bazel team provides support for the code. However, this repository is part of the test suite used to vet new Bazel releases. See {gh-path}`How to contribute ` for information on our development workflow. -## Bzlmod support - -- Status: GA +## Examples -See {gh-path}`Bzlmod support ` for any behaviour differences between -`bzlmod` and `WORKSPACE`. +This documentation is an example of `sphinxdocs` rules and the rest of the +components have examples in the {gh-path}`examples` directory. ## Migrating from the bundled rules The core rules are currently available in Bazel as built-in symbols, but this form is deprecated. Instead, you should depend on rules_python in your -`WORKSPACE` file and load the Python rules from -`@rules_python//python:defs.bzl`. +`WORKSPACE` or `MODULE.bazel` file and load the Python rules from +`@rules_python//python:defs.bzl` or load paths described in the API documentation. A [buildifier](https://github.com/bazelbuild/buildtools/blob/master/buildifier/README.md) fix is available to automatically migrate `BUILD` and `.bzl` files to add the @@ -66,13 +76,18 @@ appropriate `load()` statements and rewrite uses of `native.py_*`. buildifier --lint=fix --warnings=native-py ``` -Currently, the `WORKSPACE` file needs to be updated manually as per [Getting -started](getting-started). +Currently, the `WORKSPACE` file needs to be updated manually as per +[Getting started](getting-started). Note that Starlark-defined bundled symbols underneath `@bazel_tools//tools/python` are also deprecated. These are not yet rewritten by buildifier. +## Migrating to bzlmod + +See {gh-path}`Bzlmod support ` for any behaviour differences between +`bzlmod` and `WORKSPACE`. + ```{toctree} :hidden: From 29858d8a6650f1a8670ceb9688ec01f253fe2e88 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 13 Oct 2024 22:26:49 +0900 Subject: [PATCH 03/11] Add a note on the breaking change policy --- CONTRIBUTING.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 33b296fa64..9e8cd29895 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -194,8 +194,17 @@ The general process is: of. The API for the control mechanism can be removed in this release. Note that the `+1` and `+2` releases are just examples; the steps are not -required to happen in immedially subsequent releases. +required to happen in immediately subsequent releases. +:::{todo} +TODO: clarify this before releasing `1.0` + +Maybe: +1. In `N.M.0` we introduce the new behaviour, but it is disabled by a feature flag. +2. In `N.M+1.0` we may choose the behaviour to become the default if it is not too + disruptive. +3. In `N+1.0.0` we get rid of the old behaviour +::: ### How to control breaking changes From 665e434efa6ea18f3c730ce41c3f826946582a03 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 15 Oct 2024 10:57:20 +0900 Subject: [PATCH 04/11] Apply suggestions from groodt Co-authored-by: Greg Roodt --- BZLMOD_SUPPORT.md | 2 +- docs/getting-started.md | 2 +- docs/index.md | 12 ++++++------ 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index f7b7044a04..bf9b512c24 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -65,7 +65,7 @@ provides faster package download times and integration with the credentials help ### Extra targets in `whl_library` repos Due to how `bzlmod` is designed and the visibility rules that it enforces, it is best to use -the targets in the `whl` repos as they are and not rely on using the `annotations` API to +the targets in the `whl` repos as they do not rely on using the `annotations` API to add extra targets to so-called `spoke` repos. For alternatives that should cover most of the existing usecases please see: * {bzl:obj}`py_console_script_binary` to create `entry_point` targets. diff --git a/docs/getting-started.md b/docs/getting-started.md index 99caf33aa7..9f52243fd1 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,6 +1,6 @@ # Getting started -This doc is a simplified guide to help get started started quickly. It provides +This doc is a simplified guide to help get started quickly. It provides a simplified introduction to having a working Python program for both `bzlmod` and the older way of using `WORKSPACE`. diff --git a/docs/index.md b/docs/index.md index 8482705570..09ec6bb24b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -8,7 +8,7 @@ The core Python rules -- `py_library`, `py_binary`, `py_test`, `py_proto_library`, and related symbols that provide the basis for Python support in Bazel. -When using bazel 6, the core rules built into the Bazel binary, and the symbols +When using bazel 6 (or earlier), the core rules are bundled into the Bazel binary, and the symbols in this repository are simple aliases. On bazel 7 and above `rules_python` uses a separate Starlark implementation, see {ref}`Migrating from the Bundled Rules` below. @@ -22,21 +22,21 @@ outlined in the [support](support) page. :::{topic} PyPI integration Package installation rules for integrating with PyPI and other SimpleAPI -complying indexes. +compatible indexes. These rules are still `experimental` and the APIs might change more often than the core rules or you may experience regressions between the minor -releases. In that case, please raise tickets to the GH issues bug tracker. +releases. In that case, please raise a GH issue. ::: :::{topic} Sphinxdocs -`sphinxdocs` rules allowing users to generate documentation from bazel or +`sphinxdocs` rules allow users to generate documentation from bazel or Python source code. This is available as is and the semantic versioning used by `rules_python` does -not apply to bazel rules or the output. +not apply. ::: @@ -46,7 +46,7 @@ not apply to bazel rules or the output. code. This is available as is and the semantic versioning used by `rules_python` does -not apply to the plugin. +not apply. ::: From 7a445fa6056acd247cefce4ae0c01f7acae6e956 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 15 Oct 2024 11:19:05 -0700 Subject: [PATCH 05/11] Various typo/doc clarifications --- BZLMOD_SUPPORT.md | 4 ++++ docs/index.md | 13 +++++++------ 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index bf9b512c24..85e28acb1a 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -45,6 +45,10 @@ platforms. If you want to use the same toolchain as what WORKSPACE used, then manually register the builtin Bazel Python toolchain by doing `register_toolchains("@bazel_tools//tools/python:autodetecting_toolchain")`. + +Note that using this builtin Bazel toolchain is deprecated and unsupported. +See the {obj}`runtime_env_toolchains` docs for a replacement that is marginally +better supported. **IMPORTANT: this should only be done in a root module, and may interfere with the toolchains rules_python registers**. diff --git a/docs/index.md b/docs/index.md index 09ec6bb24b..7a3252b508 100644 --- a/docs/index.md +++ b/docs/index.md @@ -8,8 +8,8 @@ The core Python rules -- `py_library`, `py_binary`, `py_test`, `py_proto_library`, and related symbols that provide the basis for Python support in Bazel. -When using bazel 6 (or earlier), the core rules are bundled into the Bazel binary, and the symbols -in this repository are simple aliases. On bazel 7 and above `rules_python` uses +When using Bazel 6 (or earlier), the core rules are bundled into the Bazel binary, and the symbols +in this repository are simple aliases. On Bazel 7 and above `rules_python` uses a separate Starlark implementation, see {ref}`Migrating from the Bundled Rules` below. @@ -32,11 +32,12 @@ releases. In that case, please raise a GH issue. :::{topic} Sphinxdocs -`sphinxdocs` rules allow users to generate documentation from bazel or -Python source code. +`sphinxdocs` rules allow users to generate documentation using Sphinx powered by Bazel, with additional functionality for documenting +Starlark and Bazel code. -This is available as is and the semantic versioning used by `rules_python` does -not apply. +The functionality is exposed because other projects find it useful, but +it is available as is and **the semantic versioning and +compatibility policy used by `rules_python` does not apply**. ::: From b082b76adcfab9ec481216d8c307c1abb0af99d6 Mon Sep 17 00:00:00 2001 From: aignas <240938+aignas@users.noreply.github.com> Date: Wed, 16 Oct 2024 14:44:23 +0900 Subject: [PATCH 06/11] doc: add a note about public symbols --- python/pip.bzl | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/python/pip.bzl b/python/pip.bzl index a1a67200b1..44ee69d65b 100644 --- a/python/pip.bzl +++ b/python/pip.bzl @@ -17,6 +17,10 @@ This contains a set of rules that are used to support inclusion of third-party dependencies via fully locked `requirements.txt` files. Some of the exported symbols should not be used and they are either undocumented here or marked as for internal use only. + +If you are using a bazel version 7 or above with `bzlmod`, you should only care +about the {bzl:obj}`compile_pip_requirements` macro exposed in this file. The +rest of the symbols are for legacy `WORKSPACE` setups. """ load("//python/private:normalize_name.bzl", "normalize_name") From 188c47641620f0f8f16512031ba5397f4cecad76 Mon Sep 17 00:00:00 2001 From: aignas <240938+aignas@users.noreply.github.com> Date: Wed, 16 Oct 2024 14:51:10 +0900 Subject: [PATCH 07/11] chore: improve the docs --- examples/bzlmod/MODULE.bazel.lock | 4 ++-- examples/pip_parse_vendored/requirements.bzl | 3 +-- gazelle/manifest/defs.bzl | 2 +- gazelle/manifest/generate/generate.go | 2 +- internal_setup.bzl | 1 - python/extensions/pip.bzl | 8 +++++++- python/private/pypi/extension.bzl | 6 ++---- 7 files changed, 14 insertions(+), 12 deletions(-) diff --git a/examples/bzlmod/MODULE.bazel.lock b/examples/bzlmod/MODULE.bazel.lock index e292528dc0..800aeb01b8 100644 --- a/examples/bzlmod/MODULE.bazel.lock +++ b/examples/bzlmod/MODULE.bazel.lock @@ -1231,7 +1231,7 @@ }, "@@rules_python~//python/extensions:pip.bzl%pip": { "general": { - "bzlTransitiveDigest": "R4UDOUIolqXHNPsTlMq16g3t0xLeGMK/p7Ou1Cziz04=", + "bzlTransitiveDigest": "YMoII3MBUYQgraGlNXhLCnXGOv0HMK09DMpgESyf3JI=", "usagesDigest": "MChlcSw99EuW3K7OOoMcXQIdcJnEh6YmfyjJm+9mxIg=", "recordedFileInputs": { "@@other_module~//requirements_lock_3_11.txt": "a7d0061366569043d5efcf80e34a32c732679367cb3c831c4cdc606adc36d314", @@ -6138,7 +6138,7 @@ }, "@@rules_python~//python/private/pypi:pip.bzl%pip_internal": { "general": { - "bzlTransitiveDigest": "8o/oP53WcjE/4DQ4VQ0l0IVCa/i3A1mTDmN05qwm0yM=", + "bzlTransitiveDigest": "B6W/lsnMpteKi0K83y1zDkcM6X3/JBR23HYp9GWPcuY=", "usagesDigest": "Y8ihY+R57BAFhalrVLVdJFrpwlbsiKz9JPJ99ljF7HA=", "recordedFileInputs": { "@@rules_python~//tools/publish/requirements.txt": "031e35d03dde03ae6305fe4b3d1f58ad7bdad857379752deede0f93649991b8a", diff --git a/examples/pip_parse_vendored/requirements.bzl b/examples/pip_parse_vendored/requirements.bzl index 50bfe9fe8e..a64f10c27c 100644 --- a/examples/pip_parse_vendored/requirements.bzl +++ b/examples/pip_parse_vendored/requirements.bzl @@ -3,8 +3,7 @@ @generated by rules_python pip_parse repository rule. """ -load("@rules_python//python:pip.bzl", "pip_utils") -load("@rules_python//python/pip_install:pip_repository.bzl", "group_library", "whl_library") +load("@rules_python//python:pip.bzl", "group_library", "pip_utils", "whl_library") all_requirements = [ "@my_project_pip_deps_vendored_certifi//:pkg", diff --git a/gazelle/manifest/defs.bzl b/gazelle/manifest/defs.bzl index eacf1c18cf..3a65bffec4 100644 --- a/gazelle/manifest/defs.bzl +++ b/gazelle/manifest/defs.bzl @@ -39,7 +39,7 @@ def gazelle_python_manifest( manifest, meaning testing it is just as expensive as generating it, but modifying it is much less likely to result in a merge conflict. pip_repository_name: the name of the pip_install or pip_repository target. - pip_deps_repository_name: deprecated - the old pip_install target name. + pip_deps_repository_name: deprecated - the old {bzl:obj}`pip_parse` target name. manifest: the Gazelle manifest file. defaults to the same value as manifest. **kwargs: other bazel attributes passed to the generate and test targets diff --git a/gazelle/manifest/generate/generate.go b/gazelle/manifest/generate/generate.go index 19ca08a2d6..27cf2a21d8 100644 --- a/gazelle/manifest/generate/generate.go +++ b/gazelle/manifest/generate/generate.go @@ -55,7 +55,7 @@ func main() { &pipRepositoryName, "pip-repository-name", "", - "The name of the pip_install or pip_repository target.") + "The name of the pip_parse or pip.parse target.") flag.StringVar( &modulesMappingPath, "modules-mapping", diff --git a/internal_setup.bzl b/internal_setup.bzl index 3029c1546f..0e28a30e70 100644 --- a/internal_setup.bzl +++ b/internal_setup.bzl @@ -42,7 +42,6 @@ def rules_python_internal_setup(): python_versions = sorted(TOOL_VERSIONS.keys()), ) - # Because we don't use the pip_install rule, we have to call this to fetch its deps pypi_deps() bazel_skylib_workspace() diff --git a/python/extensions/pip.bzl b/python/extensions/pip.bzl index e9d47263d5..62a51c67ea 100644 --- a/python/extensions/pip.bzl +++ b/python/extensions/pip.bzl @@ -12,7 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. -"pip module extension for use with bzlmod" +""" +This is the successor to {bzl:obj}`pip_parse` for including third party PyPI dependencies into your bazel module using `bzlmod`. + +:::{seealso} +For user documentation see the [PyPI dependencies section](pypi-dependencies). +::: +""" load("//python/private/pypi:pip.bzl", _pip = "pip") diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 36fb20e030..6f8ca587c1 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -764,8 +764,7 @@ the BUILD files for wheels. attrs = _pip_parse_ext_attrs(), doc = """\ This tag class is used to create a pip hub and all of the spokes that are part of that hub. -This tag class reuses most of the pip attributes that are found in -@rules_python//python/pip_install:pip_repository.bzl. +This tag class reuses most of the attributes found in {bzl:obj}`pip_parse`. The exception is it does not use the arg 'repo_prefix'. We set the repository prefix for the user and the alias arg is always True in bzlmod. """, @@ -814,8 +813,7 @@ the BUILD files for wheels. ), doc = """\ This tag class is used to create a pypi hub and all of the spokes that are part of that hub. -This tag class reuses most of the pypi attributes that are found in -@rules_python//python/pip_install:pip_repository.bzl. +This tag class reuses most of the attributes found in {bzl:obj}`pip_parse`. The exception is it does not use the arg 'repo_prefix'. We set the repository prefix for the user and the alias arg is always True in bzlmod. """, From d16bb571564221408793dab8c598a2d022e53cd9 Mon Sep 17 00:00:00 2001 From: aignas <240938+aignas@users.noreply.github.com> Date: Thu, 17 Oct 2024 12:03:14 +0900 Subject: [PATCH 08/11] reword PyPI --- docs/index.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/index.md b/docs/index.md index 7a3252b508..378aac2faa 100644 --- a/docs/index.md +++ b/docs/index.md @@ -24,9 +24,10 @@ outlined in the [support](support) page. Package installation rules for integrating with PyPI and other SimpleAPI compatible indexes. -These rules are still `experimental` and the APIs might change more often -than the core rules or you may experience regressions between the minor -releases. In that case, please raise a GH issue. +These rules work and can be used in production, but the cross-platform building +that supports pulling PyPI dependencies for a target platform that is different +from the host platform is still in beta and the APIs that are subject to potential +change are marked as `experimental`. ::: From a517ca8d3d1e722bd7021d5a00c1cd04f5be62f0 Mon Sep 17 00:00:00 2001 From: aignas <240938+aignas@users.noreply.github.com> Date: Thu, 17 Oct 2024 12:04:22 +0900 Subject: [PATCH 09/11] document the future breaking change process --- CONTRIBUTING.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9e8cd29895..a31781a89e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -196,15 +196,11 @@ The general process is: Note that the `+1` and `+2` releases are just examples; the steps are not required to happen in immediately subsequent releases. -:::{todo} -TODO: clarify this before releasing `1.0` - -Maybe: +Once The first major version is released, the process will be: 1. In `N.M.0` we introduce the new behaviour, but it is disabled by a feature flag. 2. In `N.M+1.0` we may choose the behaviour to become the default if it is not too disruptive. -3. In `N+1.0.0` we get rid of the old behaviour -::: +3. In `N+1.0.0` we get rid of the old behaviour. ### How to control breaking changes From e9f01d6a021047513da2526d18bf6e347d97c1d3 Mon Sep 17 00:00:00 2001 From: aignas <240938+aignas@users.noreply.github.com> Date: Thu, 17 Oct 2024 12:23:56 +0900 Subject: [PATCH 10/11] revert --- examples/pip_parse_vendored/requirements.bzl | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/examples/pip_parse_vendored/requirements.bzl b/examples/pip_parse_vendored/requirements.bzl index a64f10c27c..50bfe9fe8e 100644 --- a/examples/pip_parse_vendored/requirements.bzl +++ b/examples/pip_parse_vendored/requirements.bzl @@ -3,7 +3,8 @@ @generated by rules_python pip_parse repository rule. """ -load("@rules_python//python:pip.bzl", "group_library", "pip_utils", "whl_library") +load("@rules_python//python:pip.bzl", "pip_utils") +load("@rules_python//python/pip_install:pip_repository.bzl", "group_library", "whl_library") all_requirements = [ "@my_project_pip_deps_vendored_certifi//:pkg", From 60b81c1472585ab98e333a49320e55811a048219 Mon Sep 17 00:00:00 2001 From: aignas <240938+aignas@users.noreply.github.com> Date: Fri, 18 Oct 2024 14:55:32 +0900 Subject: [PATCH 11/11] fixup - lockfile --- examples/bzlmod/MODULE.bazel.lock | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/bzlmod/MODULE.bazel.lock b/examples/bzlmod/MODULE.bazel.lock index 8c66a1318f..8cda22c3ae 100644 --- a/examples/bzlmod/MODULE.bazel.lock +++ b/examples/bzlmod/MODULE.bazel.lock @@ -1392,7 +1392,7 @@ }, "@@rules_python~//python/extensions:pip.bzl%pip": { "general": { - "bzlTransitiveDigest": "iikkSIkMsBiM/vadkEf9xEoVbaxZqrkUg08hiHr/LKk=", + "bzlTransitiveDigest": "Okeg+CvZ5figeuuknTrCNwQR1dUzHr1+YvojOpIGXEI=", "usagesDigest": "MChlcSw99EuW3K7OOoMcXQIdcJnEh6YmfyjJm+9mxIg=", "recordedFileInputs": { "@@other_module~//requirements_lock_3_11.txt": "a7d0061366569043d5efcf80e34a32c732679367cb3c831c4cdc606adc36d314", @@ -6299,7 +6299,7 @@ }, "@@rules_python~//python/private/pypi:pip.bzl%pip_internal": { "general": { - "bzlTransitiveDigest": "WPfU9gogl29lCI8A/N2aYn7RAhsCpZikVU1Hw7nMtAc=", + "bzlTransitiveDigest": "dF4Tc81oqhFbL+/QjHfkMmWICMXhiduPI6eIB5d9VJY=", "usagesDigest": "Y8ihY+R57BAFhalrVLVdJFrpwlbsiKz9JPJ99ljF7HA=", "recordedFileInputs": { "@@rules_python~//tools/publish/requirements.txt": "031e35d03dde03ae6305fe4b3d1f58ad7bdad857379752deede0f93649991b8a",