diff --git a/documentation/repository-files/license-files.md b/documentation/repository-files/license-files.md index bf0d4952..01900568 100644 --- a/documentation/repository-files/license-files.md +++ b/documentation/repository-files/license-files.md @@ -1,4 +1,4 @@ -# Your repository should have a LICENSE.md file +# License files for scientific Python open source software :::{button-link} https://www.pyopensci.org/about-peer-review/ :color: primary @@ -7,7 +7,7 @@ Want to learn how to add a license file to your GitHub repository? Check out this lesson. ::: -A license file is important for any open source project to have. The license file helps your users and the community understand +A license file is important for any open source project to have. The license file helps your users and the community understand: 1. how they can use your software 2. whether the software can be reused or adapted for other purposes @@ -17,16 +17,40 @@ and more. ## Where to store your license. -The root of your GitHub / GitLab repository should have a `LICENSE.txt` file that contains descriptive text that identifies the type of license that you have selected for your software. +The root of your GitHub / GitLab repository should have a `LICENSE` file that +contains descriptive text that identifies the type of license that you have +selected for your software. -### What type of license to use +### Use open permissive licenses when possible -If you are [submitting your project to pyOpenSci for peer review](https://www.pyopensci.org/about-peer-review/index.html), then your project should use an [Open Software Initiative (OSI) approved license ](https://opensource.org/licenses). +We generally suggest that you use a permissive, license that is [Open Software Initiative (OSI) approved](https://opensource.org/licenses). If you are +[submitting your package to pyOpenSci for peer review](https://www.pyopensci.org/about-peer-review/index.html), then we require an OSI approved +license. + +### How to chose a license To select your license, we suggest that you use GitHub's [Choose a License tool ](https://choosealicense.com/). -If you choose your license through GitHub, you can also automatically get a copy of the license file to add to your repository. +If you choose your license when creating a new GitHub repository, you can also +automatically get a text copy of the license file to add to your repo. However +in some cases the license that you want is not available through that online +process. + +:::{admonition} License recommendations from the SciPy package +[The SciPy documentation has an excellent overview of licenses.](https://docs.scipy.org/doc/scipy/dev/core-dev/index.html#licensing). Once of the key elements +that these docs recommend is ensuring that the license that you select is +complementary to license used in the core scientific Python ecosystem. +Below is a highlight of this text which outlines license that are compatible +with the modified BSD license that SciPy uses. + +> Other licenses that are compatible with the modified BSD license that SciPy uses are 2-clause BSD, MIT and PSF. Incompatible licenses are GPL, Apache and custom licenses that require attribution/citation or prohibit use for commercial purposes. + +Too coordinate with other packages in our scientific ecosystem, we also recommend +that you consider using either BSD or MIT as your +license. If you are unsure, the MIT license tends to be a simpler easier-to-understand option. +::: + ## Important: make sure that you closely follow the guidelines outlines by the License that you chose @@ -49,7 +73,7 @@ This means that technically, if you copy code from the Stack Overflow website, a 🚨 Proceed with caution! 🚨 ::: -[The SciPy documentation has an excellent license discussion that is worth reading and considering for your project's development guide.](https://docs.scipy.org/doc/scipy/dev/core-dev/index.html#licensing) + +This is just a concept - if we use change the icons to match the steps +::: :::{admonition} Learning Objectives +:class: tip In this lesson you will learn: @@ -26,12 +20,23 @@ In this lesson you will learn: - How to install your package in editable mode for interactive development To complete this lesson you will need a local Python (development) -environment. You are welcome to use any environment manager that you chose. [Review this lesson,](1-create-environment) which walks you through creating an environment using both `venv` and `conda`. If you aren't sure which environment manager to use and +environment. You are welcome to use any environment manager that you chose. + +* [If you need guidance creating a Python environment, review this lesson](extras/1-create-environment.md) which walks you through creating an environment using both `venv` and `conda`. +* If you aren't sure which environment manager to use and you are a scientist, we suggest that you use `conda`. ::: ## Make your package installable + +:::{figure-md} packages-environment + +Diagram showing .. more here if this stays. + +This is an ugly a concept. The graphic should show that you can turn your script into something that can be installed into a contained Python environment. Or you can turn your code into a full package installable from PyPI (or github but we can leave that out to reduce complexity). +::: + It’s time to create the most basic version of a Python package. While this code can't be yet published to PyPI or conda and is not documented, it will be installable on your computer or @@ -48,8 +53,7 @@ To make your code installable you need: The directory structure you’ll create in this first section looks like this: -```toml - +```bash pyospackage/ └─ pyproject.toml └─ src/ # the src directory ensures your tests always run on the installed @@ -87,7 +91,7 @@ The `__init__.py` file also: :::{admonition} The **init**.py file :class: tip -You can technically install a package without an `__init__.py` file since Python 3.3, we suggest that you include it in your package structure as it allows you to make some important customizations to your package’s user experience. +Since Python 3.3 came out, you can install a package without an `__init__.py` file. However, we suggest that you include it in your package structure as it allows you to customize your package’s user experience. ::: ### What is a pyproject.toml file? @@ -96,14 +100,12 @@ The **pyproject.toml** file is: - Where you store your project’s metadata (including its name, authors, license, etc) - Where you store dependencies (the packages that it depends on) -- Used to specify and configure what build back end you want to use to create your package. - -A **pyproject.toml**: This file is critical for both installing your package and publishing to pyPi. This is where you will declare your project build system tools, dependencies and metadata. More on that later +- Used to specify and configure what build back end you want to use to build your package distributions that are used for PyPI publication. -After the `__init__.py` and `pyproject.toml` files have been added, your package can be built and distributed as an installable Python package using tools like pip. Note that the `pyproject.toml` file needs to have the a few basic items defined for this to work including: +After the `__init__.py` and `pyproject.toml` files have been added, your package can be built and distributed as an installable Python package using tools such as pip. Note that the `pyproject.toml` file needs to have the a few basic items defined for it to be installable including: -- The build backend that you want to use, -- The project name, and a few other metadata elements. +- The build-backend that you want to use, +- The project name and version. :::{admonition} pro tip :class: tip @@ -122,11 +124,13 @@ Neither 'setup.py' nor 'pyproject.toml' found. ## Try it yourself - Create your package! -Now that you understand the basics, it's time to create a Python package! Create a directory structure similar to the structure below. If you don’t wish to create each of the files and directories below, you can always [fork and clone and customize the pyOpenSci example package, here](https://github.com/pyOpenSci/pyosPackage) +Now that you understand the basics, it's time to create a Python package! Below you will create a directory structure similar to the structure described above. + +If you don’t wish to create each of the files and directories below, you can always [fork and clone and customize the pyOpenSci example package, here.](https://github.com/pyOpenSci/pyosPackage) ### Step 1: Set Up the Package Directory Structure -Create a new directory for your package. Choose a name for your package, preferably in lowercase and without spaces (e.g., "pyospackage"). +Create a new directory for your package. Choose a name for your package, preferably in lowercase and without spaces (e.g., "pyospackage_yourname"). Inside the package directory, @@ -145,7 +149,7 @@ Your final package directory structure should look like this: pyospackage/ └─ pyproject.toml └─ src/ - └── pyospackage/ + └── pyospackage_yourname/ β”œβ”€β”€ __init__.py ``` @@ -236,8 +240,8 @@ instance, a `build-system` table most often holds 2 variables: 2. `build-backend` is used to define specific build-backend name, (in this example we are using `hatchling.build`). TOML organizes data structures, defining relationships within a configuration -file. You will learn more about the pyproject.toml format in the -[next lesson when you add additional metadata / information to this file.](/python-package-tutorial/pyproject-toml.html) +file. You will learn more about the `pyproject.toml` format in the +[next lesson when you add additional metadata / information to this file.](3-pyproject-toml.md) ```toml # An example of the build-system table which contains two variables - requires and build-backend @@ -268,7 +272,9 @@ version using git tags in the version and release your package lesson. - First open bash and `cd` into your package directory -- Activate the Python environment that you wish to use. If you need help with working with virtual environments [check out this lesson](1-create-environment). +- Activate the Python environment that you wish to use. If you need help with working with virtual environments [check out this lesson](extras/1-create-environment.md). - Finally run `python -m pip install -e .` ```bash @@ -312,12 +318,15 @@ Obtaining file:///Users/leahawasser/Documents/GitHub/pyos/pyosPackage # use pip list instead of conda list here if you are working in an venv environment rather than a conda envt ``` -Before we go any further, let's break down this command: +:::{admonition} What does `pip install -e .` do? +:class: tip + +Let's break down `pip install -e .` + -`python -m pip install -e .` (e for editable) `pip install -e .` installs your package into the current active -Python environment in **editable mode**. Installing your package in +Python environment in **editable mode** (`-e`). Installing your package in editable mode, allows you to work on your code and then test the updates interactively in your favorite Python interface. One important caveat of editable mode is that every time you update your code, you may need to restart your Python kernel. @@ -326,11 +335,47 @@ mode) you can use: - `python -m pip install . ` -:::{admonition} +:::{admonition} Using `python -m` when calling `pip` :class: tip + Above, you use`python -m` to call the version of pip installed into your current active environment. ::: +#### Look for pyospackage in your environment + +Once you have installed your package, you can view it in your current +environment. If you are using `venv` or `conda`, `pip` list will allow you +to see your current package installations. + +Note that because pyospackage is installed in editable mode (`-e`) pip will show you the past to where you package installation's code +is. + +```bash +$ pip list + +➜ pip list +Package Version Editable project location +----------------------------- -------------- -------------------------------------------------------------- +... +arrow 1.2.3 +... +... +mamba 1.1.0 +markdown-it-py 2.2.0 +MarkupSafe 2.1.2 +matplotlib 3.7.1 +msgpack 1.0.5 +mypy 1.4.1 +nox 2021.10.1 +numpy 1.24.2 +packaging 23.0 +pandas 1.5.3 +pyosPackage 0.1.0 /Users/yourusername/path/here/pyosPackage +... +... +... +``` + ### 6. Test out your new package After installing your package, type β€œpython” at the command prompt to start @@ -402,19 +447,30 @@ level vs. what makes the most sense to keep in individual modules. You did it! You have now created a Python package that you can install into any Python environment. While there is still more to do, you have completed the first major step. -In the next lesson you will: +In the upcoming lessons you will: -1. Flesh out metadata for your package within your `pyproject.toml` file. -2. Learn how to build your learn how to build your package distribution files (**sdist** and **wheel**) and -3. Learn how to publish your package on **test PyPI**. +* add a [README file](2-add-readme.md) and [LICENSE ](3-add-license-file.md) to your package +* [Add more metadata to your `pyproject.toml`](3-pyproject-toml.md) file to support PyPI publication. +* [Learn how to build your your package distribution](4-publish-pypi.md) files (**sdist** and **wheel**) and +publish to **test PyPI**. +* Finally you will learn how to publish to conda-forge from PyPI. If you have a package that is ready for the mainstream user then you can also publish your package on PyPI. +https://hatch.pypa.io/latest/intro/#existing-project <- hatch will migrate from setup.py for you > + +FROM CAROL: Probably more accurate to say that the _init_.py allows Python to recognize that the directory is a module that may be imported and used. A package can have multiple modules. + +- https://docs.python.org/3/tutorial/modules.html#packages -< link to this docs...> + +- i think it would be cool to have a package spectrum graphic. on the left is you just have code that you want to use, then the code can be installed, then you can build a package... etc etc so each tutorial would highlight a step across that spectrum +--> diff --git a/tutorials/2-add-readme.md b/tutorials/2-add-readme.md index bd861c42..b2a5aca0 100644 --- a/tutorials/2-add-readme.md +++ b/tutorials/2-add-readme.md @@ -1,10 +1,13 @@ -# What goes into your Python package README.md file? +# Add a README.md file to your Python package + + ```{button-link} https://www.pyopensci.org/python-package-guide/documentation/repository-files/readme-file-best-practices.html :color: primary :class: sd-rounded-pill float-left -If you want to learn more about readme files, you can check out our guidebook reference: the art of readme files. +Click here to learn more about best practices for creating README.md files. ``` :::{admonition} TODOS @@ -69,7 +72,7 @@ The README.md file is not only the landing page for your package, it also is lis 1. **Badges:** Add any badges below the name of your file. if you don't have badges yet, that is ok. you'll have at least one once you [publish your package to PyPI](4-publish-pypi) in lesson 4. 2. **Package overview:** Below the badges, add a section that provides an easy-to-understand overview of what your package does. Keep this section short and if you can avoid jargon or define technical words to make the description accessible to more people. 3. **Installation Instructions:** Below the description add installation instructions. this might tell people how to install your package `pip install packagename` or `conda install`... You can come back and add this information after you publish to PyPI in lesson 4. -4. **Additional Setup Information** In this section also briefly document (or link to documentation for) any additional setup that is required to use your package. This might include tokens or authentication information if it is applicable to your package. Or additional installations of tools such as GDAL, etc. Note: many packages wont need any additional information here! +4. **Additional Setup Information** In this section also briefly document (or link to documentation for) any additional setup that is required to use your package. This might include tokens or authentication information if it is applicable to your package. Or additional installations of tools such as GDAL, etc. Note: many packages won't need any additional information here! 5. **How to use your package:** Next add a brief demo of how to use your package. this might include a small code chunk that demonstrates importing and a quick call to functionality in your package. 6. **Descriptive links to docs:** Unless you already have your documentation online, you can leave this section empty for now. this section would include links to tutorials or documentation get-started pages that will help your users understand how to use your package. 7. **Community section:** this is where you'll add links to your contributing guide and code_of_conduct once you create those. You can also leave this empty for now. diff --git a/tutorials/3-add-license-file.md b/tutorials/3-add-license-file.md index 7f82653e..b9644fe8 100644 --- a/tutorials/3-add-license-file.md +++ b/tutorials/3-add-license-file.md @@ -1,4 +1,13 @@ -# potentiall drop this lesson? Add a LICENSE file to your Python package +# Add a LICENSE file to your Python package + +This lesson will be a short overview that helps people find license text. +shows them how to add a license to a new repo (but there is a create repo lesson) + +it can be short... + +we could have a small demo of adding a license when you create a new repo here? + +and then a link to the github repo where ou can find text for other licenses?? TODO: this may not be a needed lesson because that file can be added when you create a new repo. Question: can you add a license after the fact via github?? diff --git a/tutorials/3-pyproject-toml.md b/tutorials/3-pyproject-toml.md index a4326dca..820131d3 100644 --- a/tutorials/3-pyproject-toml.md +++ b/tutorials/3-pyproject-toml.md @@ -1,19 +1,24 @@ # Make your Python package PyPI ready - pyproject.toml -:::{admonition} TODOS + -In [the installable code lesson](2-installable-code), you learned how to add the bare minimum information to a `pyproject.toml` file to make it pip installable. +In [the installable code lesson](2-installable-code), you learned how to add the bare minimum information to a `pyproject.toml` file to make it `pip` installable. You then learned + +* How to add a user-friendly `README.md` file to your package and +* How to add a `LICENSE` file -To both publish your package to PyPI and also help users find your package, you will want to add additional metadata to your `pyproject.toml` file that describes the use and contents of your package. You will +To both publish your package to PyPI and also help users find your package, +you will want to add additional metadata to your `pyproject.toml` file that +describes the use and contents of your package. You will learn how to do that in this lesson. +build and then publish to PyPI and conda --> - + +:::{admonition} Learn more about building Python packages in our guide +:class: tip - [Learn more about building here](../package-structure-code/python-package-distribution-files-sdist-wheel) - [Learn more about the wheel](../package-structure-code/python-package-distribution-files-sdist-wheel#wheel-whl-files) - [Learn more about the sdist (source distribution)](../package-structure-code/python-package-distribution-files-sdist-wheel#source-distribution-sdist) +::: + ## Step 1: Build your package's sdist and wheel distributions -To begin you will need to create your package's sdist and wheel distribution -files. +To begin you will need to create your package's sdist and wheel distribution +files. -1. You will build your package locally and then -2. Push your package to PyPI using twine. +1. You will build your package locally and then +2. Push your package to PyPI using twine. -The first time you publish your package to PyPI you will do it manually. -After that you can opt to create a automated workflow that publishes an updated -version of your package to PyPI every time you create a release on GitHub. +The first time you publish your package to PyPI you will do it manually. +After that you can opt to create a automated workflow that publishes an updated +version of your package to PyPI every time you create a release on GitHub. -1. activate your development environment if it is not already active. -2. install your package requirements if you haven't already. based on the pypoject toml +1. Activate your development environment if it is not already active. +2. Install your package requirements if you haven't already. based on the `pyproject.toml` (do i need this if they just install twine manually below? ) + +??????!!!! - TODO: add a dev dependencies with build and twine? +`pip install build` -`pip install -e .[dev]` 3. You are now ready to build your package! Note that here you are using the [PyPA build tool](https://github.com/pypa/build) as a "Front end" tool that builds -your package's sdist and wheel using the hatchling build back end. Remember that you defined your build backend here in the build system table of your pyproject.toml file. So build knows to use [hatchling](https://hatch.pypa.io/latest/). +your package's sdist and wheel using the hatchling build back end. Remember that you defined your build backend here in the build system table of your `pyproject.toml` file. So build knows to use [hatchling](https://hatch.pypa.io/latest/). To build your package run: `python -m build` @@ -100,9 +106,14 @@ To build your package run: * Building wheel... Successfully built pyospackage-0.1.0.tar.gz and pyospackage-0.1.0-py3-none-any.whl ``` -When you build your package, it will create two output "files". A .whl or wheel file and a .tar.gz compressed file. +When you build your package, it will create two output "files": + +1. [.whl or wheel file](/package-structure-code/python-package-distribution-files-sdist-wheel.html#wheel-whl-files) and +2. a [sdist (source distribution)](../package-structure-code/python-package-distribution-files-sdist-wheel.html#source-distribution-sdist) stored in a `.tar.gz` compressed format. + +You can learn more about both of these distribution files and package +building in general in the [build page of our packaging guide](../package-structure-code/python-package-distribution-files-sdist-wheel). -You can learn more about both of these distribution files here. ``` dist/pyospackage-0.1.0-py3-none-any.whl dist/pyospackage-0.1.0.tar.gz @@ -110,31 +121,231 @@ dist/pyospackage-0.1.0.tar.gz The above two files are what you will need to publish your package to PyPI. The tar.gz file is particularly important if you wish to publish your package to conda-forge. You'll learn more about that in this lesson. -### Time to celebrate! +### Time to celebrate! + +You've now created your package distribution! You're officially on +your way to publishing your package on PyPI. + +## 2. Setup your testPyPI account + +Next, you'll setup an account on `testPyPI`. You are setting things up on testPyPi in this lesson as a way to safely learn how to publish a package to PyPI within filling the real PyPI's servers up with lots of "test" packages. + +:::{admonition} Test vs. real PyPI +If you have a package that you are confident belongs on the real PyPI. All of the steps below will also work for you with slight modifications which will be noted below. +::: + +1. [Open up a web browser and go to the test PyPI website](https://test.pypi.org/). +2. [Create an account](https://test.pypi.org/account/register/) if you don't already have one. Be sure to store your password in a safe place! +3. Once you have an account setup, login to it. +3. Search on https://test.pypi.org/ (or pypi.org) to ensure that the package name that you have selected doesn't already exist. If you are using our test pyosPackage, then we suggest that you add your name or github username to the end of the package name to ensure it's unique. Example: `pyosPackage_yourNameHere`. + + +:::{figure-md} build-workflow-tutorial +alt here. + +caption here +::: + + +:::{admonition} Setup 2-factor (2FA) authentication + +2-factor authentication is a secure login process that allows you to +use a backup device that only you can access to validate that the person logging in is really you. It addresses the issue of password phishing where someone else gains access to a password and can login to your account. + +This matters on PyPI because someone could login to your account and upload a version of your package that has security issues. These issues will then impact all of your users when they download and install that version of the package. + +While you don't have to setup 2-factor authentication, we strongly +suggest that you do so. +::: + + + +## 3. Create a package upload token + +To upload your package to PyPI, you will need to create a token. Ideally +this token is specific to the package that you are publishing. + +However, if your package isn’t already on PyPI, then you will need to create a token for your account first and then create a package-specific token. + +:::{admonition} Why create package-specific tokens? +It's ideal to create a package-specific token as it will ensure that you don't why??? TODO <- > +::: + + +## Create a secure PyPI token + +Follow the steps below to create your token. + +* First login and go to your account settings in testPyPI +* Scroll down to the **API tokens** section +* Click on the **Add API Token** button +* If you are new to using PyPI and don't have any packages there yet, OR if you have other packages on PyPI but are uploading a new package, you will need to create an account-wide token. + + +:::{admonition} tools and stuff +In this lesson you use twine to upload to PyPI. Note that other tools such as Flit, Hatch and PDM use slightly different approaches to storing and accessing your token to support uploading your package to PyPI. +::: + + + + +### Using a PyPI token + +To use your API token, you do the following: + +* Set your username to __token__ +* Set your password to the token value, including the pypi- prefix + +### Upload to PyPI using twine + +Here we use Twine to upload your project to PyPI. Twine allows you to store both your username and token value in a `.pypirc` file stored in your `$HOME` directory. This file is then used every time you upload to PyPI so you don't have to find and enter the long token each time. + +```toml +[pypi] +username = __token__ +password = +``` + +NOTE: the `.pypirc` file stores your token in plain text format. Thus, in another lesson you will learn how to store your PyPI authentication information in a GitHub secret / or trusted publisher ... -you've now created your package distrbution! you're on your way to publishing your package on PyPI. - -## Step 2: +### Create your `.pypirc` file with authentication information +To create your `.pypirc` file, do the following. -Next, you'll setup an account on testPyPI. we are using testPyPi for now as a way to safely learn how to publish a package to PyPI within filling the real PyPI up with lots of "test" packages used to learn. +1. open up bash / your terminal +2. Create a new file using touch +3. Open the file in your favorite text editor. Below you use vscode but any text editor will work. + +The commands for the above steps are below: + +```bash +$ touch ~/.pypirc +$ code .pypirc # open with vscode +``` +Once you open up the file in vscode or whatever text editor you prefer, you can add a toml table containing your testPyPI username and token information stored in a password key. + +```toml +[testpypi] + username = __token__ + password = pypi-rest-of-your-token-here +``` +Save the file and then check that it looks correct in bash using the `cat` command: + +```bash +# view the file printed in your terminal +$ cat ~/.pypirc +``` + +Now, install twine: + +```bash +python3 -m pip install --upgrade twine +``` + +### Upload your package + +Once you have stored your PyPI authentication information you can +upload your package to PyPI. + +```bash + +❯ python3 -m twine upload --repository testpypi dist/* + +Uploading distributions to https://test.pypi.org/legacy/ +Uploading pyospackage-0.1.0-py3-none-any.whl +100% ━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 kB β€’ 00:00 β€’ 6.6 MB/s +Uploading pyospackage-0.1.0.tar.gz +100% ━━━━━━━━━━━━━━━━━━━━━━━━━ 8.6/8.6 kB β€’ 00:00 β€’ 3.1 MB/s + +View at: +https://test.pypi.org/project/pyosPackage/0.1.0/ +``` + +(TODO: pypa has windows and mac/unix instructions they are slightly different) + +:::{admonition} Two factor authentication and PyPI + +If you have 2FA setup and try to upload to PyPI using a username and +password, you will get an error similar to the one below. To +upload to PyPI you will need to setup a token authentication. +You will learn how to do that below. + +```bash +➜ python3 -m twine upload --repository testpypi dist/* +Uploading distributions to https://test.pypi.org/legacy/ +Enter your username: your-pypi-username-here +Enter your password: your-password-here +Uploading pyospackage-0.1.0-py3-none-any.whl +100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 kB β€’ 00:00 β€’ 4.0 MB/s +WARNING Error during upload. Retry with the --verbose option for more details. +ERROR HTTPError: 401 Unauthorized from https://test.pypi.org/legacy/ + User lwasser has two factor auth enabled, an API Token or Trusted Publisher must + be used to upload in place of password. +``` -:::{admonition} Test vs real pypi -IF you have a package that you are confident belongs on the real PyPI. All of the steps below will also work for you with slight modifications which will be noted below. ::: -1. [go to test pypi](https://test.pypi.org/) and [create an account](https://test.pypi.org/account/register/) if you don't already have one. If you have an account login to it now. -2. Search on PyPI to ensure that the package name that you have selected doesn't already exist. If you are using our test pyosPackage, then we suggest that you add your name or github username to the end of the package name to ensure it's unique. Example: pyosPackage_yourNameHere. +## Install your package + +Once your package is uploaded to test PyPI, you can install it. +You can find the installation instructions on the testPyPI landing page. + +[Check out our pyOpenSci test package landing page here](https://test.pypi.org/project/pyosPackage/) for an example. NOtice at the top there is the package name and version. Below are installation instructions: + +`pip install -i https://test.pypi.org/simple/ pyosPackage` + +On your computer, activate the development environment that +you wish to install your newly published package in. +Then run the installation instructions for your package from PyPI. +``` +conda activate pyospkg-dev +#venv... +`pip install -i https://test.pypi.org/simple/ youPackageNameHere` +``` + + + +we can then setup trusted publisher for actions... +https://pypi.org/help/#twofa + +> from PyPI here - https://pypi.org/help/#apitoken - You can create a token for an entire PyPI account, in which case, the token will work for all projects associated with that account. Alternatively, you can limit a token's scope to a specific project. + + +## Don't forget to create a package-specific token +If you plan to use your token regularly to publish to PyPI, we strongly recommend going through the above steps again to create +a token specific to your new package. + +To do this: +1. Go to testPyPI +1. Navigate to the "Your Projects" section of your account +2. Click on the manage button for the project that ou wish to add a token for +3. Go to settings +4. Click on "Create a token for your-package-name-here" +5. Create the token and follow the steps above to open up the `.pypirc` file +6. Replace the old account wide token with your new package token. + +And you're all done! + +:::{admonition} The value of end-to-end tools like hatch, flit and poetry +In this lesson we are using core tools including: + +* hatchling +* PyPA's build +* twin + +to build and publish your package to PyPI. End to end packaging tools +such as Hatch, PDM, Poetry and Flit can manage all of the above steps but have to be configured properly. ... Maybe more about why some people might like these tools and others prefer just using core tools... + +end-to-end build tools such as hatch will cache your token information for you link to hatch docs... +::: + +## You have published your package to (test) PyPI! -Set password -You can set the password locally if you wish to publish locally. To do this +Congratulations. You have now successfully published your package to testPyPI. If you have a package that is ready for real-world use on the real pyPi, then you can follow the same steps to publish it on PyPI.org . -2. Create a pypi account - Create a new token - if the package isn’t already on pypi you may have to create a token for your account at first before you publish - Once you have created that token and successfully uploaded the project to pypi, you can then create a project specific token for your package. - If you do that be sure to go back after and create one for just your package… +Once you publish on PyPI.org, you can then easily add your package to the conda-forge ecosystem using the [grayskull](https://conda-forge.org/blog/posts/2020-03-05-grayskull/) tool. -pdm config repository.pypi.password +`__init__.py` -Start with test pypi for experience +You will learn how to do that in the next lesson. diff --git a/tutorials/intro.md b/tutorials/intro.md index ac4feddf..1a9b9804 100644 --- a/tutorials/intro.md +++ b/tutorials/intro.md @@ -1,39 +1,61 @@ -# What is a Python package? +# Python packaging 101 + -- sometimes people might start by creating a new repo and cloning it. -* Other times they might start by working locally and then pushing the repo content to github / gitlab. in any case the readme and license lessons may or may not be relevant. so we should have a pregame for setting up a fresh repo, creating an initial .gitignore, readme and license. and then link to the chose a license lesson. +Welcome to the pyOpenSci Python packaging tutorial series. The lessons on the upcoming pages walk you through the core steps needed to +create a Python package. + + +:::{figure-md} packaging-outline + +Diagram showing .. more here if this stays. +TODO: this will not be the final graphic but i want to provide the user within a roadmap. i'm intentionally splitting out each topic into lessons so people can also chose to select a topic they are having issues with rather than going through the entire thing looking for a specific topic. ::: -This lesson is a part of a set. +The content in this tutorial series is beginner friendly and assumes that you have not created a Python package before! With that said, you may find the content useful to understand the various elements of packaging even if you are not a total beginner. -```{toctree} -:hidden: -:caption: Pre game -:glob: -extras/* -``` + +:::{admonition} Lesson Outline +:class: important + +Below are the lessons contained in the packaging 101 series. In this series you will learn about the core elements that you need to publish your package to PyPI. + + +* What is a Python package? +* [Make your code pip installable](1-installable-code.md) +* Add a README file +* Add a LICENSE file +* Make your package PyPI ready +* Publish your package to PyPI +* Publish your package to Conda Forge + +In the second series, you will learn about infrastructure and documentation needed to support package maintenance. +::: + ```{toctree} :hidden: -:caption: Packaging 101 +:caption: Python Packaging 101 :glob: What is a Python package? +Make your code pip installable <1-installable-code> * ``` -:::{admonition} Learning Objectives +## What is a Python package? -## Learning objectives +:::{admonition} Learning Objectives +This lesson introduces you to the basic components of a Python package. After reading this lesson you will: - Understand what a Python package is @@ -42,33 +64,99 @@ After reading this lesson you will: ::: -## What is a Python package? - A Python package is a collection of related modules / code containing functions, classes and methods that are organized together in a directory. Packages allow you to group and structure your Python code, making it easier to manage and reuse code across different projects. At a high level, you can think about each package as a toolbox filled with different tools that perform specific actions in your code when imported and called. -:::{figure-md} toolbox +:::{figure-md} python-toolbox -Diagram showing .. more here if this stays. +Diagram showing ADD ALT. You can think about a package as a set of tools. A tool may be a function or a class. Each tool does a specific thing well. ::: -### The code in your package should be generalizable -Ideally the code in your Python package is generalizable. This means it + + + +## The elements of a Python package + +:::{figure-md} package-components + +Diagram showing .. more here if this stays. + +more here - might turn this into something more linear? +::: + +The core elements of Python package include: + +- **Code:** Generalized code that performs operations that you may need to complete multiple times +- **Documentation:** documentation with tutorials / examples that help users get started using your tool + - Documentation also helps people to contribute to your package. +- **Tests:** that makes sure your code works as it should and makes it easier for you and others to contribute to, modify and update the code in the future +- **License:** An open source license …. - link to choose a license… +- **Infrastructure** that automates updates, publication workflows and runs test suites + +If you intend for others to use and contribute to your code, consider who will maintain it over time. You will want a **contributing / development** guide to help new potential contributors get started with contributing to your package. And a **code of conduct** to ensure community interactions remain healthy both for you and your contributors / maintainer team + +_Link to our EiC checklist for core components that pyOpenSci looks for… _ + +A package in any language is more than just code. If your package is public facing, meaning people besides yourself will use it, you will want to think about the various elements of a package that make it a useful community resource. + +Most Python packages live in an online version control platform such as GitHub or GitLab. These platforms support robust infrastructure including continuous integration and continuous deployment (CI/CD). + +:::{figure-md} packaging-workflow + +Diagram showing .. more here if this stays. + +The parts of a Python package... can this diagram be simplified a bit or at least +use simpler language?? maybe it's like code, docs, metadata, tests, all packaged up in a specific structure that tools can understand and turn into an installable package. that simpler version might be better. +::: + +>then in the publish section below we can add the arrow with that structure being +>built into distributions that get published on pypi and conda... + +>then in the community part we add another section that is the github repo with users contributing... very simple diagrams... + + + + + +```{toctree} +:hidden: +:caption: Pre game +:glob: +extras/* + +``` + + +## What should code in a Python package look like? + +Ideally the code in your Python package is general. This means it can be applied in different settings. -An example of a package that has a generalized scope is matplotlib. It does +An example of a package that is written in a generalized way is `matplotlib`. +`matplotlib` does one (big important) thing really well: It creates visual plots of data. -`Matplotlib` is used by thousands of users for a host of different plotting applications. While few scientific packages will have the same broad application as tools like `matplotlib` or `numpy`, the +`Matplotlib` is used by thousands of users for different plotting applications +using different types of data. While few scientific packages will have the same broad application as tools like `matplotlib` or `numpy`, the idea of code being used for something more than a single workflow still applies. :::{admonition} Where do research compendia fit in? @@ -86,7 +174,7 @@ which supports a specific project. ::: -### Packages and environments +### Python packages and environments A Python package can be installed into a Python environment in the same way you might install `numpy` or `pandas`. Installing your package into an environment @@ -96,9 +184,44 @@ allows you to access it from any code run with that specific Python environment Diagram showing .. more here if this stays. -Caption here +This is just a concept - the graphic should show that you can turn your script into something that can be installed into a Python environment. Or you can turn your code into a package... ::: +## Publishing a package + +If you want to make your package installable using `pip` you will want to +publish it in a repository such as PyPI or conda-forge. If you package is a +pure python package, then luckily publish to both takes just a few steps. + +First, you will learn how to publish to PyPI using twine. +Then you can create a conda-forge recipe using the grayskull tool to easily publish to conda-forge from PyPI. you will learn more about the conda-forge publication process here. *TODO add link when lesson is created * + +You will learn how +to publish to both [PyPI](4-publish-pypi.md) and conda forge in this tutorial series. + +:::{figure-md} publish-package + +Diagram showing .. more here if this stays. + +TODO: make the flower simpler because not all of that goes into the published +output. make sure to add use grayskull to publish to conda-forge from PyPI. +::: + +## Yay, your package has users! Now what? + +As the community using your package grows, you may also find yourself managing users, contributors and others who want to interact with your package. It’s important to consider all this before you dive into development. Once you have a user base in the community, people will depend upon your code to work and will need direction regarding how to use it. + +To support your community, you'll want to add things like: + +* development guide +* code of conduct +* contributing guide + + + -- **Code:** Generalized code that performs operations that you may need to complete multiple times -- **Documentation:** documentation with tutorials / examples that help users get started using your tool - - Documentation also helps people to contribute to your package. -- **Tests:** that makes sure your code works as it should and makes it easier for you and others to contribute to, modify and update the code in the future -- **License:** An open source license …. - link to choose a license… -- **Infrastructure** that automates updates, publication workflows and runs test suites -If you intend for others to use and contribute to your code, consider who will maintain it over time. You will want a **contributing / development** guide to help new potential contributors get started with contributing to your package. And a **code of conduct** to ensure community interactions remain healthy both for you and your contributors / maintainer team +***** -_Link to our EiC checklist for core components that pyOpenSci looks for… _ - -## Publishing a package - -- publish to pypi -- if you want to publish to conda... there are tools that make this easy.. explain how it works and link to the publish tutorial -## Yay, your package has users! Now what? - -As the community using your package grows, you may also find yourself managing users, contributors and others who want to interact with your package. It’s important to consider all this before you dive into development. Once you have a user base in the community, people will depend upon your code to work and will need direction regarding how to use it. + ## What's next?