diff --git a/documentation/repository-files/code-of-conduct-file.md b/documentation/repository-files/code-of-conduct-file.md index 71c205ab..c5cbc502 100644 --- a/documentation/repository-files/code-of-conduct-file.md +++ b/documentation/repository-files/code-of-conduct-file.md @@ -1,13 +1,13 @@ -# The CODE_OF_CONDUCT.md file In Your Python Open Source Package +# The CODE_OF_CONDUCT file - Python Packaging -```{admonition} Example Code of Conduct files +```{admonition} Example CODE_OF_CONDUCT files :class: tip * [SciPy Code of Conduct file - notice they included theirs in their documentation](https://docs.scipy.org/doc/scipy/dev/conduct/code_of_conduct.html) -* [fatiando code of conduct file](https://github.com/fatiando/community/blob/main/CODE_OF_CONDUCT.md) +* [fatiando CODE_OF_CONDUCT.md file](https://github.com/fatiando/community/blob/main/CODE_OF_CONDUCT.md) ``` -Your package should have a **CODE_OF_CONDUCT.md** file located +Your package should have a `CODE_OF_CONDUCT.md` file located the root of the repository. Once you have people using your package, you can consider the package itself as having a community around it. Some of this community uses your tool. These users @@ -17,20 +17,20 @@ Others in the community might want to contribute to your tool. They might fix bugs, update documentation and engage with the maintainer team. -## Why you need a code of conduct +## Why you need a CODE_OF_CONDUCT In order to keep this community healthy and to protect yourself, -your mainatianer team and your users from unhealthy behavior, -it is important to have a [code of conduct](https://opensource.guide/code-of-conduct/). +your maintainer team and your users from unhealthy behavior, +it is important to have a [`CODE_OF_CONDUCT`](https://opensource.guide/code-of-conduct/). -The code of conduct is important +The `CODE_OF_CONDUCT` is important as it establishes what you expect in terms of how users and contributors interact with maintainers and each other. It also establishes rules and expectations which can then be enforced -if need be to protect others from harmful and/or negative behaviors. +if need be to protect others from harmful and/or negative behaviors. If you are not comfortable -with creating your own code of conduct text, we encourage you to adopt the -code of conduct language used in the [Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/). -[Many other communities](https://www.contributor-covenant.org/adopters/) have adopted this code of conduct as +with creating your own `CODE_OF_CONDUCT` text, we encourage you to adopt the +`CODE_OF_CONDUCT` language used in the [Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/). +[Many other communities](https://www.contributor-covenant.org/adopters/) have adopted this `CODE_OF_CONDUCT` as their own. See the [Fatiando a Terra Geoscience Python community's example here.](https://github.com/fatiando/community/blob/main/CODE_OF_CONDUCT.md) diff --git a/documentation/repository-files/license-files.md b/documentation/repository-files/license-files.md index 01900568..83525f26 100644 --- a/documentation/repository-files/license-files.md +++ b/documentation/repository-files/license-files.md @@ -7,19 +7,33 @@ Want to learn how to add a license file to your GitHub repository? Check out this lesson. ::: -A license file is important for any open source project to have. The license file helps your users and the community understand: +## What is a Open Source License file? -1. how they can use your software -2. whether the software can be reused or adapted for other purposes -3. how people can (or can't) contribute to your project +When we talk about LICENSE files, we are referring to a file in your +GitHub or GitLab repository that contains legally binding language +that describes to your users how they can legally use (and not use) your package. + +## Why licenses are important + +A license file is important for all open source projects because it protects both you as a maintainer and your users. The license file helps your users and the community understand: + +1. How they can use your software +2. Whether the software can be reused or adapted for other purposes +3. How people can contribute to your project and more. -## Where to store your license. +[Read more about why license files are critical in protecting both you as a maintainer and your users of your scientific Python open source package.](https://opensource.guide/legal/#just-give-me-the-tldr-on-what-i-need-to-protect-my-project) + -The root of your GitHub / GitLab repository should have a `LICENSE` file that -contains descriptive text that identifies the type of license that you have -selected for your software. +## Where to store your license + +Your `LICENSE` file should be stored at root of your GitHub / GitLab repository. + +Some maintainers customize the language in their license files for specific reasons. However, if you are just getting started, we suggest that you select a +permissive license and then use the legal language templates provided both by GitHub and/or the [choosealicense.com](https://choosealicense.com/) website. + +Licenses are legally binding, as such you should avoid trying to create your own license unless you have the guidance of legal council. ### Use open permissive licenses when possible @@ -27,7 +41,7 @@ We generally suggest that you use a permissive, license that is [Open Software I [submitting your package to pyOpenSci for peer review](https://www.pyopensci.org/about-peer-review/index.html), then we require an OSI approved license. -### How to chose a license +### How to choose a license To select your license, we suggest that you use GitHub's [Choose a License tool ](https://choosealicense.com/). @@ -46,7 +60,7 @@ with the modified BSD license that SciPy uses. > Other licenses that are compatible with the modified BSD license that SciPy uses are 2-clause BSD, MIT and PSF. Incompatible licenses are GPL, Apache and custom licenses that require attribution/citation or prohibit use for commercial purposes. -Too coordinate with other packages in our scientific ecosystem, we also recommend +To coordinate with other packages in our scientific ecosystem, we also recommend that you consider using either BSD or MIT as your license. If you are unsure, the MIT license tends to be a simpler easier-to-understand option. ::: @@ -74,11 +88,13 @@ This means that technically, if you copy code from the Stack Overflow website, a ::: +## What about software citation? - diff --git a/images/tutorials/environment-package-install.png b/images/tutorials/environment-package-install.png new file mode 100644 index 00000000..d2f131a4 Binary files /dev/null and b/images/tutorials/environment-package-install.png differ diff --git a/images/tutorials/package-components.png b/images/tutorials/package-components.png index 123b4d46..4741b2ec 100644 Binary files a/images/tutorials/package-components.png and b/images/tutorials/package-components.png differ diff --git a/images/tutorials/publish-package-pypi-conda.png b/images/tutorials/publish-package-pypi-conda.png new file mode 100644 index 00000000..345d3037 Binary files /dev/null and b/images/tutorials/publish-package-pypi-conda.png differ diff --git a/images/tutorials/test-pypi-package.png b/images/tutorials/test-pypi-package.png new file mode 100644 index 00000000..b1861cd2 Binary files /dev/null and b/images/tutorials/test-pypi-package.png differ diff --git a/images/tutorials/view-license-github.png b/images/tutorials/view-license-github.png new file mode 100644 index 00000000..57dc8d2a Binary files /dev/null and b/images/tutorials/view-license-github.png differ diff --git a/package-structure-code/pyproject-toml-python-package-metadata.md b/package-structure-code/pyproject-toml-python-package-metadata.md index 729f9fd0..03227fe8 100644 --- a/package-structure-code/pyproject-toml-python-package-metadata.md +++ b/package-structure-code/pyproject-toml-python-package-metadata.md @@ -65,7 +65,7 @@ Prior to August 2017, Python package metadata was stored either in the `setup.py - allows you to use a suite of different [build backends](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-build-tools.html#build-back-ends) such as (flit-core, hatchling, pdm-build), and - aligns with modern best practices. -If you are migrating from a **setup.py** or **setup.cfg** file, and want help, [check out this tutorial.](6-setup-py-to-pyproject-toml) +If you are migrating from a **setup.py** or **setup.cfg** file, and want help, [check out this tutorial.](../tutorials/extras/6-setuppy-to-pyproject-toml.md) ::: The standard file that Python packages use to [specify build requirements and diff --git a/package-structure-code/python-package-distribution-files-sdist-wheel.md b/package-structure-code/python-package-distribution-files-sdist-wheel.md index 0af8f50f..19886502 100644 --- a/package-structure-code/python-package-distribution-files-sdist-wheel.md +++ b/package-structure-code/python-package-distribution-files-sdist-wheel.md @@ -145,7 +145,7 @@ or is more complex in its build, the two distributions will be very different. Also note that we are not discussing conda build workflows in this section. [You can learn more about conda builds here.](https://conda.io/projects/conda-build/en/latest/user-guide/tutorials/index.html) ``` - +(python-source-distribution)= ## Source Distribution (sdist) **Source files** are the unbuilt files needed to build your @@ -219,6 +219,7 @@ items including a metadata directory and if you use `setuptools_scm` or `hatch_v the sdist may also contain a file that stores the version. ``` +(python-wheel)= ## Wheel (.whl files): A wheel file is a ZIP-format archive whose filename follows a specific format diff --git a/tutorials/1-installable-code.md b/tutorials/1-installable-code.md index 1a735776..cb4c861a 100644 --- a/tutorials/1-installable-code.md +++ b/tutorials/1-installable-code.md @@ -1,12 +1,18 @@ # Make your Python code pip installable +The first step in creating a Python package based on code that you +have is to make that code pip installable. You will learn how to make +your code pip installable in this lesson. + :::{figure-md} code-to-script Diagram showing .. more here if this stays. -This is just a concept - if we use change the icons to match the steps +It might be cool to make another version of this with add license and readme file. and end with build and publish to PyPI. then after that pip install from PyPI This is just a concept - if we use change the icons to match the steps ::: :::{admonition} Learning Objectives @@ -32,16 +38,19 @@ you are a scientist, we suggest that you use `conda`. :::{figure-md} packages-environment -Diagram showing .. more here if this stays. +Diagram showing .. more here if this stays. -This is an ugly a concept. The graphic should show that you can turn your script into something that can be installed into a contained Python environment. Or you can turn your code into a full package installable from PyPI (or github but we can leave that out to reduce complexity). +Making your code pip installable is the first step towards creating a Python package. Once it is pip installable, you can add it to any Python environment on your computer and import that package in the same way that you might import a package such as Pandas or Geopandas. ::: +## Make a basic Python package + It’s time to create the most basic version of a Python package. While this code can't be yet published to PyPI or conda and is not documented, it will be installable on your computer or anyone elses. +### What does a basic package directory structure look like? To make your code installable you need: - A `pyproject.toml` file @@ -49,27 +58,25 @@ To make your code installable you need: - A specific directory structure - Some code. -### What does a basic package directory structure look like? - The directory structure you’ll create in this first section looks like this: ```bash pyospackage/ └─ pyproject.toml - └─ src/ # the src directory ensures your tests always run on the installed - └── pyospackage/ # package directory where code lives, use the package name + └─ src/ # The src directory ensures your tests always run on the installed + └── pyospackage/ # Package directory where code lives, use the package name ├── __init__.py ├── add_numbers.py - └──# Add any other .py modules that you want here + └── # Add any other .py modules that you want here ``` -If you already know what all of the elements of this package structure are, you can skip to the next lesson. Otherwise keep reading to learn about each element of the above structure. +Below, you will learn about each element of the above package structure. -### Above the above package directory structure +### About the basic package directory structure Notice a few things about the above layout: -1. Your package code lives within a `src/packagename` directory. While it’s fine if you wish to use a [flat layout](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#about-the-flat-python-package-layout) (a flat layout does not have a src/ directory at the root), we suggest that you use `src/` directory as it ensure you are running tests on the installed version of your code. [Learn more here.](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#the-src-layout-and-testing) +1. Your package code lives within a `src/packagename` directory. We suggest that you use `src/` directory as it ensure you are running tests on the installed version of your code. However, you are welcome to instead use a [flat layout](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#about-the-flat-python-package-layout) which does not have a src/ directory at the root. [Learn more here.](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#the-src-layout-and-testing) 2. Within the `src/` directory you have a package directory called `pyospackage/`. Use the name of your package for that directory name. 3. In your package directory, you have an `__init__.py` file and all of your Python modules. 4. The `pyproject.toml` file lives at the root directory of your package. @@ -241,7 +248,7 @@ instance, a `build-system` table most often holds 2 variables: TOML organizes data structures, defining relationships within a configuration file. You will learn more about the `pyproject.toml` format in the -[next lesson when you add additional metadata / information to this file.](3-pyproject-toml.md) +[next lesson when you add additional metadata / information to this file.](5-pyproject-toml.md) ```toml # An example of the build-system table which contains two variables - requires and build-backend @@ -437,6 +444,8 @@ level vs. what makes the most sense to keep in individual modules. :::{important} +TODO: We had a bit of discussion in slack about this and i'd like to +get more feedback via a review before fleshing this section out more. i'm still unclear on when this should be taught and what guidelines to provide as to when to surface functionality in a package... > TODO: Guidelines for when and how to add methods and such to the `__init__.py` file here. > `__all__ = ['add_num']` @@ -449,9 +458,9 @@ You did it! You have now created a Python package that you can install into any In the upcoming lessons you will: -* add a [README file](2-add-readme.md) and [LICENSE ](3-add-license-file.md) to your package -* [Add more metadata to your `pyproject.toml`](3-pyproject-toml.md) file to support PyPI publication. -* [Learn how to build your your package distribution](4-publish-pypi.md) files (**sdist** and **wheel**) and +* add a [README file](2-add-readme.md) and [LICENSE ](4-add-license-file.md) to your package +* [Add more metadata to your `pyproject.toml`](5-pyproject-toml.md) file to support PyPI publication. +* [Learn how to build your your package distribution](6-publish-pypi.md) files (**sdist** and **wheel**) and publish to **test PyPI**. * Finally you will learn how to publish to conda-forge from PyPI. diff --git a/tutorials/2-add-readme.md b/tutorials/2-add-readme.md index b2a5aca0..50066589 100644 --- a/tutorials/2-add-readme.md +++ b/tutorials/2-add-readme.md @@ -1,37 +1,69 @@ -# Add a README.md file to your Python package +# Add a README file to your Python package +TODO: +* add link - + +* add a screenshot of the pyospackage readme rendered in GitHub +* this series would be great if there was a graphic with all of the steps. and each was highlighted at the top of the page. +* **TODO:** we didn't ask them to put code into a src dir in the previous lesson? probably good to do that. + + --> + + + +In the [previous lesson](1-installable-code.md) you: + +1. Learned how create the basic structure of a Python package and +2. How to make your code `pip` installable. + +:::{admonition} Learning objectives + +In this lesson you will learn: + +1. How to add a **README.md** file to your package. +2. What the core elements of a **README.md** file are. + +::: + + +::::{grid} 2 + +:::{grid-item} +:columns: 4 +:class: sd-m-auto ```{button-link} https://www.pyopensci.org/python-package-guide/documentation/repository-files/readme-file-best-practices.html -:color: primary +:color: secondary :class: sd-rounded-pill float-left -Click here to learn more about best practices for creating README.md files. + pyOpenSci README file overview guide. ``` -:::{admonition} TODOS -:class: important - -> this series would be great if there was a graphic with all of the steps. and each was highlighted at the top of the page. -> **TODO:** we didn't ask them to put code into a src dir in the previous lesson? probably good to do that. -> ... - ::: -In the previous lesson you: +:::{grid-item} +:columns: 4 +:class: sd-m-auto -1. learned how create the basic structure of a Python package and -2. how to make your code `pip` installable. +```{button-link} https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-readmes +:color: secondary +:class: sd-rounded-pill float-left -:::{admonition} Learning objectives + Guide: about `README.md` files +``` +::: -In this lesson you will learn: -1. How to add a README.md file to your package. -2. What the core elements of a readme file are. +:::{grid-item} +:columns: 4 +:class: sd-m-auto +empty ::: +:::: + + ## What is a README file? @@ -41,19 +73,19 @@ a user understand: - You package's name and what it does - The current development "state" of the package (through badges) - How to use your package: this might include a short get started demo that shows someone how to import and quickly use your package. Or it could be a set of links to get started tutorials in your documentation. -- How to contribute to your package: normally you would link to a `CONTRIBUTING.md` and `Code_of_conduct` file in this part of the readme +- How to contribute to your package: normally you would link to a `CONTRIBUTING.md` and `CODE_OF_CONDUCT.md` file in this part of the readme - How to cite your package -Your **README.md** file is important as it is often the first thing that someone sees before they install your package. The README file also will be used to populate your PyPI landing page. So it's good to create this file before publishing to PyPI. +Your **README.md** file is important as it is often the first thing that someone sees before they install your package. The README file also will be used to populate your PyPI landing page. So it's good to create this file before you publish to PyPI. -### GitHub and readme files. +### GitHub highlights your README file as a core file -Every GitHub repository landing page has a right hand side bar. This sidebar lists the elements of your package including there is column on the right hand side that lists and links to core informational elements of your package including: +Every GitHub repository landing page has a right hand side bar that lists and links to elements of your package including: -- the Readme file -- License -- starts and -- forks +- A link to your Readme file +- Your license +- Number of Stars and +- Number of Forks :::{figure-md} github-sidebar Add alt @@ -63,20 +95,20 @@ The README.md file is not only the landing page for your package, it also is lis ## Create a README.md file for your package -- To get started if it doesn't already exist, create a file called `README.md` in your local github repository. +- To get started if it doesn't already exist, create a file called `README.md` in your local GitHub repository. - At the top of the `README.md` file, add the name of your package (and a logo if you have one). It's fine if you don't have a logo. - Next add the following sections to your `README.md` file -1. **Badges:** Add any badges below the name of your file. if you don't have badges yet, that is ok. you'll have at least one once you [publish your package to PyPI](4-publish-pypi) in lesson 4. +1. **Badges:** Add any badges below the name of your file. if you don't have badges yet, that is ok. you'll have at least one once you [publish your package to PyPI](6-publish-pypi) in lesson 4. 2. **Package overview:** Below the badges, add a section that provides an easy-to-understand overview of what your package does. Keep this section short and if you can avoid jargon or define technical words to make the description accessible to more people. 3. **Installation Instructions:** Below the description add installation instructions. this might tell people how to install your package `pip install packagename` or `conda install`... You can come back and add this information after you publish to PyPI in lesson 4. 4. **Additional Setup Information** In this section also briefly document (or link to documentation for) any additional setup that is required to use your package. This might include tokens or authentication information if it is applicable to your package. Or additional installations of tools such as GDAL, etc. Note: many packages won't need any additional information here! 5. **How to use your package:** Next add a brief demo of how to use your package. this might include a small code chunk that demonstrates importing and a quick call to functionality in your package. 6. **Descriptive links to docs:** Unless you already have your documentation online, you can leave this section empty for now. this section would include links to tutorials or documentation get-started pages that will help your users understand how to use your package. -7. **Community section:** this is where you'll add links to your contributing guide and code_of_conduct once you create those. You can also leave this empty for now. -8. **Citation information:** also leave this empty for now. You can learn how to setup your repository with zenodo to make it citable in [this lesson](extras/2-connect-repo-to-zenodo.md) in a future lesson. +7. **Community section:** this is where you'll add links to your contributing guide and `CODE_OF_CONDUCT.md` once you create those. You can also leave this empty for now. You will add a [`CODE_OF_CONDUCT.md` file in the next lesson](4-add-license-file.md). +8. **Citation information:** also leave this empty for now. You can learn how to setup your repository with zenodo to make it citable in [this lesson](extras/2-connect-repo-to-zenodo.md). Your finished `README.md` file should look something like this: @@ -114,7 +146,7 @@ You can also add any links to this section to tutorials in your documentation. ## Community -information here about contributing to your package. links to your code of conduct and development guide. +Add information here about contributing to your package. Be sure to add links to your `CODE_OF_CONDUCT` file and your development guide. For now this section might be empty. You can go back and fill it in later. ## How to cite packagename @@ -123,12 +155,17 @@ citation information here ## Wrapping up -While there are no hard and fast rules dictating what a README file looks like, -it's important to consider the information that a new user or contributor might -need when creating it. Above is a set of recommendations as you are just getting +It's important to consider the information that a new user or contributor might +need when creating your `README.md` file. Above is a set of recommendations as you are just getting started. You may find the need for other elements to be added to this file as you further develop your package and as a community begins to use your package. -In the next lesson, you will add another critical file - a LICENSE file - to -your Python package repository. +For instance, as you add get-started tutorials to your documentation, you can +link directly to those in the README file. + +In the [next lesson](4-add-license-file.md), you will add a LICENSE file to +your Python package repository. A license file is critical as it tells users how they legally can (and can't use your package). It also: + +* Builds trust with your users +* Discourages misuse of your package and associated code diff --git a/tutorials/3-add-license-file.md b/tutorials/3-add-license-file.md deleted file mode 100644 index b9644fe8..00000000 --- a/tutorials/3-add-license-file.md +++ /dev/null @@ -1,45 +0,0 @@ -# Add a LICENSE file to your Python package - -This lesson will be a short overview that helps people find license text. -shows them how to add a license to a new repo (but there is a create repo lesson) - -it can be short... - -we could have a small demo of adding a license when you create a new repo here? - -and then a link to the github repo where ou can find text for other licenses?? - -TODO: this may not be a needed lesson because that file can be added when you create a new repo. -Question: can you add a license after the fact via github?? - -If you created your package repository on GitHub, then you may have already -selected a license to use and thus may already have a LICENSE file in your -repository. If that is the case, you can skip this lesson. - -- Where do most people start? if they start by creating a new github repo, the license and readme are a part of the options. - -https://opensource.guide/legal/#just-give-me-the-tldr-on-what-i-need-to-protect-my-project - -This tutorial walks you through adding a LICENSE file to your GitHub repository. - -:::{button-link} https://www.pyopensci.org/python-package-guide/documentation/repository-files/license-files.html -:color: primary -:class: sd-rounded-pill float-left - -To learn more about what a license file is and why they are important -in open source software, check out our guidebook page -::: - -## What is a license? - -Very brief overview here. - -### How to chose a license - -- add steps here around selecting a license using the choose a license tool. -- suggest a license if they want guidance. - -:::{figure-md} github-new-repo -Add alt - -::: diff --git a/tutorials/4-add-license-file.md b/tutorials/4-add-license-file.md new file mode 100644 index 00000000..6d19090f --- /dev/null +++ b/tutorials/4-add-license-file.md @@ -0,0 +1,110 @@ +# Add a LICENSE & CODE_OF_CONDUCT to your Python package + +In the [previous lesson](2-add-readme) you created a basic `README.md` file for your scientific Python package's repository. + +:::{admonition} Learning objectives +:class: tip + +In this lesson you will learn: + +1. How to select and add a `LICENSE` file to your package repository with a focus on the GitHub interface. +2. How to add a `CODE_OF_CONDUCT` file to your package repository. +3. How you can use the Contributors Covenant website to add generic language as a starting place for your `CODE_OF_CONDUCT`. +::: + + + +## Add a LICENSE file to your repository + +A license file is a document that contains legal language about how users are allowed to use and reuse your software. Generally, we suggest that you select a permissive license that accommodates the other most commonly used licenses in the scientific Python ecosystem (MIT and BSD-3). + +[Click here for an overview of license recommendations for the scientific Python ecosystem](../documentation/repository-files/license-files.html#use-open-permissive-licenses-when-possible) + +### Where should the license file live & how do you add it? + +Your `LICENSE` file should be placed at the root of your package's repository. + +There are two ways to add a license file: + +1. When you create a new repository on GitHub, it will ask you if you wish to add a `LICENSE` file at that time. +2. You can also always go back and add a `LICENSE` file to your repo. + +### How to add a LICENSE to your GitHub repository + +If you don't already have a `LICENSE` file in your repo, add one now. + +* Follow the instructions to select and add a license to your repository on the [ GitHub LICENSE page]( https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-license-to-a-repository) . +* Once you have added your LICENSE file, be sure to sync your git local repository with the repository on GitHub.com. This means running `git pull` to update your local branch. + +:::{admonition} An overview of LICENSES in the scientific Python ecosystem +:class: note + +In the pyOpenSci [packaging guidebook](../documentation/repository-files/license-files.html), we provide an overview of license in the scientific Python ecosystem. We review why license files are important, which ones are most commonly used for scientific software and how to select the correct license. + +If you want a broad overview of why licenses are important for protecting open source software, [check out this blog post that overviews the legal side of things.](https://opensource.guide/legal/#just-give-me-the-tldr-on-what-i-need-to-protect-my-project) +::: + +:::{figure-md} github-new-repo +Add alt + +When you initially create a repository on GitHub you can add a license +through their interface. +::: + + +:::{figure-md} view-license +Add alt + +You can also view an overview of the license on GitHub if you view it in the GitHub interface. +::: + +Next, you will add a `CODE_OF_CONDUCT` file to your repository. + +## Add a CODE_OF_CONDUCT file to your repo + +A `CODE_OF_CONDUCT` file is critical to supporting your community as +grows around your project. The `CODE_OF_CONDUCT` is important as it: +1. Establishes guidelines for how users and contributors interact with each other and you in your software repository. +2. Identifies negative behaviors that you don't want in your interactions. In extreme situations, you can use this language as a moderation tool that can be references when moderating tense conversations. + +If you are unsure of what language to add to your `CODE_OF_CONDUCT` +file, we suggest that you adopt the [contributor covenant language](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) as a starting place. + +[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](#) + +* Add a `CODE_OF_CONDUCT.md` file to your repository if it doesn't +already exist. +* Visit the contributor covenant website and [add the language here to the file.](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) [A markdown version can be found here.](https://www.contributor-covenant.org/version/2/1/code_of_conduct/code_of_conduct.md) + + +:::{admonition} Additional Code of Conduct resources +:class: note + +* [ Guide: `CODE_OF_CONDUCT.md` files](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-code-of-conduct-to-your-project) +* [pyOpenSci package guide `CODE_OF_CONDUCT.md` overview](https://www.pyopensci.org/python-package-guide/documentation/repository-files/code-of-conduct-file.html) + +::: + +## Sync your LICENSE & COC files to GitHub / GitLab + +It's time to sync your repository to ensure that all of the files that you have added locally are on GitHUB.com and all of the files that have been added on GitHub are local. + +* If you added a `LICENSE` file to GitHub using the GitHub online interface, be sure to pull it down locally using `git pull`. +* Similarly be sure to pull or push your newly added `CODE_OF_CONDUCT` to your GitHub repository if you created that file locally. + +## Wrap up + +In this lesson and the [last lesson](2-add-readme), you have now added: + +* `README` file +* `LICENSE` file +* `CODE_OF_CONDUCT` file + +These are core files that every scientific Python package should include. In the next lessons, you will: +* [Flesh out your `pyproject.toml` file](5-pyproject-toml) which will support building and publishing your package on PyPI. and +* You will learn how to [publish your package to PyPI](6-publish-pypi)! diff --git a/tutorials/3-pyproject-toml.md b/tutorials/5-pyproject-toml.md similarity index 89% rename from tutorials/3-pyproject-toml.md rename to tutorials/5-pyproject-toml.md index 820131d3..9b4f43a1 100644 --- a/tutorials/3-pyproject-toml.md +++ b/tutorials/5-pyproject-toml.md @@ -11,10 +11,10 @@ license = { text = "MIT" } that format is a toml inline table https://toml.io/en -In [the installable code lesson](2-installable-code), you learned how to add the bare minimum information to a `pyproject.toml` file to make it `pip` installable. You then learned +In [the installable code lesson](2-installable-code), you learned how to add the bare minimum information to a `pyproject.toml` file to make it `pip` installable. You then learned: * How to add a user-friendly `README.md` file to your package and -* How to add a `LICENSE` file +* How to add a `LICENSE` and `CODE_OF_CONDUCT` file To both publish your package to PyPI and also help users find your package, you will want to add additional metadata to your `pyproject.toml` file that @@ -37,9 +37,9 @@ In this lesson you will learn: If you wish to learn more about the `pyproject.toml` format, [check out this page. ](../package-structure-code/pyproject-toml-python-package-metadata.md) ::: + - -:::{admonition} Lesson highlights +:::{admonition} Lesson highlights TL&DR When creating your pyproject.toml file, consider the following: @@ -60,8 +60,8 @@ When creating your pyproject.toml file, consider the following: The `pyproject.toml` file is a human and machine-readable TOML file. The `pyproject.toml` file tells your build tool: -- What build backend to use to build your package (e.g. `setuptools`, `hatchling`, `pdm.build`, etc) -- How and where to retrieve your package's version (dynamically vs statically) +- What build backend to use to build your package (e.g. `setuptools`, `hatchling`, `pdm.build`, etc.) +- How and where to retrieve your package's version (dynamically vs. statically) - What dependencies you are using - What versions of Python your package supports @@ -114,7 +114,7 @@ repository to quickly understand: - What it does, who maintains it - And more -The file is also often used to configure tools such as static type checkers (e.g. mypy), linters (eg black, ruff), and other tools. +The file is also often used to configure tools such as static type checkers (e.g. mypy), linters (e.g. black, ruff), and other tools. ## Time to update your pyproject.toml file @@ -149,13 +149,11 @@ While the example above uses the `hatchling` build back-end you can chose to use `setuptools` or any other tool you wish to create and build your package. - - [Learn more about packaging tools and build back-ends here](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-build-tools.html#build-back-ends) +Important: In some instances, specific build back-ends will need metadata provided in a slightly different way in your `pyproject.toml` file. Be sure to read documentation for the build tool that you are using if you don't use Hatchling which is used in this tutorial. -* TODO: In some instances specific back-ends will need metadata provided in a slightly different way. be sure to read documentation for the build tool that you are using . -The documentation to use the hatchling back end is [here](https://hatch.pypa.io/latest/config/metadata/) +The documentation for the hatchling back-end is [here](https://hatch.pypa.io/latest/config/metadata/) ::: ## Add metadata to your pyproject.toml `[project]` table @@ -275,7 +273,7 @@ Pinnning dependencies references to specifying a specific version of a dependenc Note that unless you are building an application, you want to be cautious about pinning dependencies. This is because users will be installing your package into various environments. A pinned dependency can make resolving an environment more challenging. As such only pin dependencies to a specific version if you absolutely need to do so. -_TODO: Add a link to where we talk about pinning in the guide??_ +One build tool that you should be aware of that pins dependencies by default is Poetry. [Read more about how to safely add dependencies with Poetry, here.](../package-structure-code/python-package-build-tools.html#challenges-with-poetry) ::: ### Requires-python @@ -318,7 +316,7 @@ dependencies = ["ruamel-yaml>=0.17.21", "requests", "python-dotenv", "pydantic"] ### Add a license -In the previous lessons, you added both a [README.md](3-add-license-file) file and a [LICENSE](3-add-license-file) to your package repository. +In the previous lessons, you added both a [README.md](4-add-license-file) file and a [LICENSE](4-add-license-file) to your package repository. Once you have those files, you can add them to your pyproject.toml file as links following the example below. @@ -393,9 +391,9 @@ license = { text = "MIT" } ``` -## More info +## Appendix - A fully commented pyproject.toml file -possibly delete this?? it was added because it was requested. however pypa does have something like this it just it a bit challenging to follow +Below is a fully commented pyproject.toml file if you want to use it for reference. ```toml # You can delete all of the comments once you have created your own pyproject.toml file. @@ -469,18 +467,30 @@ license = { FILE = LICENSE } #package-dir = "src" -# TODO: update this to work with hatchling -# # Note that you need to create the tag after all commits are created - otherwise +# Note that you need to create the tag after all commits are created for vcs based versioning to work [tool.hatch] version.source = "vcs" build.hooks.vcs.version-file = "src//version.py" ``` -## TODO: list some example pyprpoject.toml files +## Example `pyproject.toml` files + +Below are some examples of `pyproject.toml` files from various packages in the scientific and pyOpenSci ecosystem. +* [PyPAs fully documented pyProject.toml file](https://github.com/pypa/sampleproject/blob/main/pyproject.toml) +* [taxpasta has a nicely organized pyProject.toml file and is a pyOpenSci approved package](https://github.com/taxprofiler/taxpasta/blob/f9f6eea2ae7dd08bb60a53dd49ad77e4cf143573/pyproject.toml) + -from projects in our ecosystem?? -* https://github.com/taxprofiler/taxpasta/blob/f9f6eea2ae7dd08bb60a53dd49ad77e4cf143573/pyproject.toml -* PyPAs d -* hatch documentation for metadata (which is what we want to follow using hatchling) +## What's next?? + +You are now ready to publish your Python package to PyPI and conda-forge. + +* In the next lesson, you will learn [how to publish to PyPI](6-publish-pypi). +* Following that you'll learn how to create the recipe needed for conda-forge to build your package from PyPI and [publish it on conda-forge](7-publish-conda-forge). + + diff --git a/tutorials/4-publish-pypi.md b/tutorials/6-publish-pypi.md similarity index 53% rename from tutorials/4-publish-pypi.md rename to tutorials/6-publish-pypi.md index f1d5c0d1..6fa7d33b 100644 --- a/tutorials/4-publish-pypi.md +++ b/tutorials/6-publish-pypi.md @@ -1,18 +1,5 @@ # Publish your Python package to PyPI -:::{figure-md} build-workflow-tutorial -Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to testPyPI or the real pypi. from PyPI you can then connect to conda forge for an automated build that sends distributions from PyPI to conda-forge. - -You need to build your Python package in order to publish it to PyPI (or Conda). The build process organizes your code and metadata into a distribution format that can be uploaded to PyPI and subsequently downloaded and installed by users. NOTE: you need to publish a sdist to PyPI in order for conda-forge to properly build your package automatically. -::: - - In the previous lessons, you've learned the following: @@ -22,79 +9,135 @@ In the previous lessons, you've learned the following: If you have gone through all of the above lessons, you are now ready to build your package's distribution files which are needed for you to publish -to PyPI. Here, you will learn how to set things up on -PyPI and how to manually publish to (test) PyPI using twine. +to PyPI. Here, you will learn: -You can follow the same steps to setup your package on the real PyPI. However -in this lesson we will use testPyPI as a practice run. +* How to set up your account and package on PyPI and +* How to manually publish to (test) PyPI using [twine](https://twine.readthedocs.io/en/stable/). + +In a followup lesson, you will learn how to automate publishing to PyPI using GitHub actions. -in xx lesson, you will learn how to setup an automated release workflow on GitHub -using GitHub actions that will automate the PyPI publication process whenever -you create a new software release. :::{admonition} Learning Objectives :class: tip In this lesson you will learn how to: -- how to build your package's sdist and wheel distributions +- How to build your package's sdist and wheel distributions - Setup an account on testPyPI (the process is similar for the real PyPI) -- Publish your package to PyPI +- Publish your package to testPyPI Once your package is on PyPI you can then easily publish it to conda-forge using the [grayskull](https://conda.github.io/grayskull/) tool. You do not need to build the package specifically -for conda, conda-forge will build from your PyPI source distribution file (sdist). +for conda, conda-forge will build from your PyPI source distribution file (sdist). You will learn how to publish to conda-forge in the [next lesson](7-publish-conda-forge.md). + +::: +:::{figure-md} build-workflow-tutorial +Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to testPyPI or the real pypi. from PyPI you can then connect to conda forge for an automated build that sends distributions from PyPI to conda-forge. + +You need to build your Python package in order to publish it to PyPI (or Conda). The build process organizes your code and metadata into a distribution format that can be uploaded to PyPI and subsequently downloaded and installed by users. NOTE: you need to publish a sdist to PyPI in order for conda-forge to properly build your package automatically. ::: + + +## Test PyPI.org vs PyPI + +There are two "warehouses" that you can use to publish your package. + +1. **Test PyPI (https://test.pypi.org):** Test PyPI is a version of the PyPI repository that can be used for testing. This is a great place to practice and learn how to publish a package without taking up space on the real PyPI servers. +2. **Real PyPI (https://pypi.org):** This is the PyPI "warehouse" where you can officially publish your Python package. IMPORTANT: only publish your package to PyPI when you are ready for it to be used by others and/or confident that it will become a package that you maintain. PyPI is not a place to practice learning how to publish a Python package. + +The steps for publishing on test PyPI vs. real PyPI are the same with the +exception of a different url. Thus, in this lesson you will use testPyPI +to practice and learn. + + + + ## 3 Steps for publishing a Python package on PyPI There are 3 things that you need to do to publish your Python package to PyPI. You need to: -1. [build your package](../package-structure-code/python-package-distribution-files-sdist-wheel). Building a package is the process of turning your code into 2 distribution files - called an sdist and a wheel. These files are what PyPI needs to publish your package . -2. You need to create an account on (test) PyPI if you don't already have one and a token which provides permissions for you to upload your package. -3. Once you have both of the above done, you are ready to use `twine` to publish your package! +1. [**Build your package**](../package-structure-code/python-package-distribution-files-sdist-wheel). Building a package is the process of turning your code into 2 distribution files: an sdist and a wheel. The wheel distribution file is particularly important for users who will `pip install` your package. +2. **Create an account on (test) PyPI**: You will need to create a PyPI account and associated token which provides permissions for you to upload your package. +3. **Publish to PyPI using `twine`**: Once you have completed the above two steps, you are ready to use `twine` to publish your package! - - - +The first time you publish your package to PyPI you will do it manually. +After that you can opt to create a automated workflow that publishes an updated +version of your package to PyPI every time you create a release on GitHub. :::{admonition} Learn more about building Python packages in our guide :class: tip - [Learn more about building here](../package-structure-code/python-package-distribution-files-sdist-wheel) -- [Learn more about the wheel](../package-structure-code/python-package-distribution-files-sdist-wheel#wheel-whl-files) -- [Learn more about the sdist (source distribution)](../package-structure-code/python-package-distribution-files-sdist-wheel#source-distribution-sdist) +- [Learn more about the wheel](#python-wheel) +- [Learn more about the sdist (source distribution)](#python-source-distribution) ::: ## Step 1: Build your package's sdist and wheel distributions To begin you will need to create your package's sdist and wheel distribution -files. - -1. You will build your package locally and then -2. Push your package to PyPI using twine. - -The first time you publish your package to PyPI you will do it manually. -After that you can opt to create a automated workflow that publishes an updated -version of your package to PyPI every time you create a release on GitHub. +files. This process is known as building your package. 1. Activate your development environment if it is not already active. 2. Install your package requirements if you haven't already. based on the `pyproject.toml` (do i need this if they just install twine manually below? ) -??????!!!! - TODO: add a dev dependencies with build and twine? -`pip install build` +### **If you are using conda for your environment manager:** + +```bash +> conda activate pyospackage-dev +> conda list +> pip install -e .[dev] +``` + +### If you are using venv: + +::::{tab-set} + +:::{tab-item} Linux/Mac +```bash +> source pyospackage-dev /bin/activate +> pip list +> pip install -e .[dev] +``` + +::: + +:::{tab-item} Windows + +```bash +> pyospackage-dev\Scripts\activate +> pip list +> pip install -e .[dev] +``` +::: + +:::: + + + + 3. You are now ready to build your package! Note that here you are using the [PyPA build tool](https://github.com/pypa/build) as a "Front end" tool that builds your package's sdist and wheel using the hatchling build back end. Remember that you defined your build backend here in the build system table of your `pyproject.toml` file. So build knows to use [hatchling](https://hatch.pypa.io/latest/). + To build your package run: `python -m build` ```bash ❯ python -m build + * Creating virtualenv isolated environment... * Installing packages in isolated environment... (hatchling) * Getting build dependencies for sdist... @@ -111,24 +154,33 @@ When you build your package, it will create two output "files": 1. [.whl or wheel file](/package-structure-code/python-package-distribution-files-sdist-wheel.html#wheel-whl-files) and 2. a [sdist (source distribution)](../package-structure-code/python-package-distribution-files-sdist-wheel.html#source-distribution-sdist) stored in a `.tar.gz` compressed format. -You can learn more about both of these distribution files and package -building in general in the [build page of our packaging guide](../package-structure-code/python-package-distribution-files-sdist-wheel). ``` dist/pyospackage-0.1.0-py3-none-any.whl dist/pyospackage-0.1.0.tar.gz ``` +:::{admonition} Learn more about the wheel and sdist formats +:class: tip +You can learn more about both of these distribution files and package +building in general in the [build page of our packaging guide](../package-structure-code/python-package-distribution-files-sdist-wheel). +::: -The above two files are what you will need to publish your package to PyPI. The tar.gz file is particularly important if you wish to publish your package to conda-forge. You'll learn more about that in this lesson. +The above two files are what you will need to publish your package to +PyPI. The tar.gz file is particularly important if you wish to [publish +your package to conda-forge](7-publish-conda-forge). You'll learn more about that in this +lesson. -### Time to celebrate! +### Congratulations - you've created your package distribution files -You've now created your package distribution! You're officially on +You've now created your package distribution. You're officially on your way to publishing your package on PyPI. ## 2. Setup your testPyPI account -Next, you'll setup an account on `testPyPI`. You are setting things up on testPyPi in this lesson as a way to safely learn how to publish a package to PyPI within filling the real PyPI's servers up with lots of "test" packages. +Next, you'll setup an account on `testPyPI`. Remember that you +are using testPyPI here instead of the real PyPI as a way to +safely learn how to publish a package without stressing the +real PyPI's servers up with lots of "test" packages. :::{admonition} Test vs. real PyPI If you have a package that you are confident belongs on the real PyPI. All of the steps below will also work for you with slight modifications which will be noted below. @@ -137,13 +189,15 @@ If you have a package that you are confident belongs on the real PyPI. All of th 1. [Open up a web browser and go to the test PyPI website](https://test.pypi.org/). 2. [Create an account](https://test.pypi.org/account/register/) if you don't already have one. Be sure to store your password in a safe place! 3. Once you have an account setup, login to it. -3. Search on https://test.pypi.org/ (or pypi.org) to ensure that the package name that you have selected doesn't already exist. If you are using our test pyosPackage, then we suggest that you add your name or github username to the end of the package name to ensure it's unique. Example: `pyosPackage_yourNameHere`. +4. Search on [https://test.pypi.org/](https://test.pypi.org/) (or pypi.org) to ensure that the package name that you have selected doesn't already exist. If you are using our test pyosPackage, then we suggest that you add your name or GitHub username to the end of the package name to ensure it's unique. +Example: `pyosPackage_yourNameHere`. :::{figure-md} build-workflow-tutorial -alt here. +This is a screenshot of the test PyPI website. At the top in the search bar, you can see the search for pyosPackage. The search return says there were no results for pyosPackage Did you mean probpackage -caption here +Before you try to upload to (test) PyPI, check to see if the name of your package is already taken. You can do that using +the search box at the top of the (test) PyPI website. ::: @@ -158,8 +212,6 @@ While you don't have to setup 2-factor authentication, we strongly suggest that you do so. ::: - - ## 3. Create a package upload token To upload your package to PyPI, you will need to create a token. Ideally @@ -168,7 +220,8 @@ this token is specific to the package that you are publishing. However, if your package isn’t already on PyPI, then you will need to create a token for your account first and then create a package-specific token. :::{admonition} Why create package-specific tokens? -It's ideal to create a package-specific token as it will ensure that you don't why??? TODO <- > + +It's ideal to create a package-specific token. When you create an account wide token this allows anyone with access to then access all of your PyPI projects. By creating a package specific token, you are limiting the scope of the token to only your specific package. This is just a safe way to set things up for you particularly if you are collaborating with others on package development. ::: @@ -182,23 +235,27 @@ Follow the steps below to create your token. * If you are new to using PyPI and don't have any packages there yet, OR if you have other packages on PyPI but are uploading a new package, you will need to create an account-wide token. -:::{admonition} tools and stuff -In this lesson you use twine to upload to PyPI. Note that other tools such as Flit, Hatch and PDM use slightly different approaches to storing and accessing your token to support uploading your package to PyPI. -::: - +:::{admonition} Tool options for publishing to PyPI +In this lesson you use twine to upload to PyPI. Note that other tools such as Flit, Hatch and PDM use slightly different approaches to storing and accessing your token to support uploading your package to PyPI. All of those tools are great ways to publish your package and provide a more end-to-end packaging solution. +::: ### Using a PyPI token To use your API token, you do the following: -* Set your username to __token__ +* Set your username to `__token__` * Set your password to the token value, including the pypi- prefix ### Upload to PyPI using twine -Here we use Twine to upload your project to PyPI. Twine allows you to store both your username and token value in a `.pypirc` file stored in your `$HOME` directory. This file is then used every time you upload to PyPI so you don't have to find and enter the long token each time. +It can be challenging to always have to type in your username +and token value for PyPI. Twin allows you to store both your +username and token value in a `.pypirc` file stored in your +computer's `$HOME` directory. This file is then used every +time you upload to PyPI so you don't have to find and enter +the long token each time. ```toml [pypi] @@ -206,7 +263,11 @@ username = __token__ password = ``` +:::{admonition} +:class: note NOTE: the `.pypirc` file stores your token in plain text format. Thus, in another lesson you will learn how to store your PyPI authentication information in a GitHub secret / or trusted publisher ... +::: + ### Create your `.pypirc` file with authentication information @@ -238,14 +299,30 @@ $ cat ~/.pypirc Now, install twine: +::::{tab-set} + +:::{tab-item} Linux/Mac + ```bash python3 -m pip install --upgrade twine ``` +::: + +:::{tab-item} Windows +```bash +py -m pip install --upgrade twine +``` +::: +:::: ### Upload your package -Once you have stored your PyPI authentication information you can -upload your package to PyPI. +Once you have stored your PyPI authentication information you +can upload your package to PyPI. + +::::{tab-set} + +:::{tab-item} Linux/Mac ```bash @@ -261,7 +338,29 @@ View at: https://test.pypi.org/project/pyosPackage/0.1.0/ ``` -(TODO: pypa has windows and mac/unix instructions they are slightly different) +::: + +:::{tab-item} Windows + +```bash + +❯ py -m twine upload --repository testpypi dist/* + +Uploading distributions to https://test.pypi.org/legacy/ +Uploading pyospackage-0.1.0-py3-none-any.whl +100% ━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 kB • 00:00 • 6.6 MB/s +Uploading pyospackage-0.1.0.tar.gz +100% ━━━━━━━━━━━━━━━━━━━━━━━━━ 8.6/8.6 kB • 00:00 • 3.1 MB/s + +View at: +https://test.pypi.org/project/pyosPackage/0.1.0/ +``` +::: + +:::: + + + :::{admonition} Two factor authentication and PyPI @@ -286,33 +385,77 @@ ERROR HTTPError: 401 Unauthorized from https://test.pypi.org/legacy/ ::: -## Install your package +## Install your package from TestPyPI + +Once your package is uploaded to test PyPI, you can install it from testPYPI. +You can find the installation instructions on the testPyPI landing page for your newly uploaded package. + +:::{figure-md} testpypi-landing-page +Add Alternatively. + +This is an example landing page for the pyosPackage that was just uploaded. Notice at the top of the page there is instructions for how to install the package from test PyPI. You can simply copy that code and use it to install your package from testPyPi locally. +::: -Once your package is uploaded to test PyPI, you can install it. -You can find the installation instructions on the testPyPI landing page. [Check out our pyOpenSci test package landing page here](https://test.pypi.org/project/pyosPackage/) for an example. NOtice at the top there is the package name and version. Below are installation instructions: `pip install -i https://test.pypi.org/simple/ pyosPackage` -On your computer, activate the development environment that +### Time to install your package + +* On your computer, activate the development environment that you wish to install your newly published package in. -Then run the installation instructions for your package from PyPI. +* Run the installation instructions for your package from test PyPI. + +::::{tab-set} + +:::{tab-item} Conda + +```bash +> conda activate pyospkg-dev +> pip install -i https://test.pypi.org/simple/ youPackageNameHere +> conda list +``` +::: + +:::{tab-item} venv mac / Linux +```bash +> source pyospackage-dev /bin/activate +> pip install -i https://test.pypi.org/simple/ youPackageNameHere +> pip list ``` -conda activate pyospkg-dev -#venv... -`pip install -i https://test.pypi.org/simple/ youPackageNameHere` +::: + +:::{tab-item} venv windows +```bash +> source pyospackage-dev /bin/activate +> pip install -i https://test.pypi.org/simple/ youPackageNameHere +> pip list ``` +::: +:::: + -we can then setup trusted publisher for actions... + + +## Package-specific token vs trusted publisher +For long run maintenance of your package, you have two options +related to PyPI publication. -## Don't forget to create a package-specific token +1. You can create a package-specific token which you will use to publish your package (manually) to PyPI. This is a great option if you don't wish to automate your PyPI publication workflow. +2. You can also create an automated publication workflow on GitHub using GitHub actions. This is a great way to make the publication process easier and it also supports a growing maintainer team. In this case we suggest you don't worry about the token and instead setup a specific GitHub action that publishes your package when you make a release. You can then create a "trusted publisher" workflow on PyPI. + +You will learn how to create the automated trusted publisher workflow in a followup lesson. + + +### If you want to use a manual token-based publication workflow If you plan to use your token regularly to publish to PyPI, we strongly recommend going through the above steps again to create a token specific to your new package. @@ -332,12 +475,15 @@ In this lesson we are using core tools including: * hatchling * PyPA's build -* twin +* twine + +to build and publish your package to PyPI. -to build and publish your package to PyPI. End to end packaging tools -such as Hatch, PDM, Poetry and Flit can manage all of the above steps but have to be configured properly. ... Maybe more about why some people might like these tools and others prefer just using core tools... +End-to-end packaging tools such as Hatch, PDM, Poetry and +Flit can manage all of the above steps but have to be +configured. -end-to-end build tools such as hatch will cache your token information for you link to hatch docs... +For example, while twine users a `.pypirc` file, Hatch will cache your PyPI token information to make publishing to PyPI from your computer easier. Be sure the read the documentation for any end-to-end publication tool that you chose to use. ::: ## You have published your package to (test) PyPI! diff --git a/tutorials/7-publish-conda-forge.md b/tutorials/7-publish-conda-forge.md new file mode 100644 index 00000000..4604f8ef --- /dev/null +++ b/tutorials/7-publish-conda-forge.md @@ -0,0 +1,49 @@ +# Publish to conda-forge + +In the previous lessons, you've learned the following: + +1. How to structure your code into a package like format that can be installed into a Python environment. +2. How to add a `README` and `LICENSE` file to your package +3. How to setup your `pyproject.toml` file with all of the metadata that PyPI requires and also metadata that will be helpful for users to find your package. + +If you have gone through all of the above lessons, you are now ready to +build your package's distribution files which are needed for you to publish +to PyPI. Here, you will learn how to set things up on +PyPI and how to manually publish to (test) PyPI using twine. + +:::{figure-md} build-workflow-tutorial +Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to testPyPI or the real pypi. from PyPI you can then connect to conda forge for an automated build that sends distributions from PyPI to conda-forge. + +You need to build your Python package in order to publish it to PyPI (or Conda). The build process organizes your code and metadata into a distribution format that can be uploaded to PyPI and subsequently downloaded and installed by users. NOTE: you need to publish a sdist to PyPI in order for conda-forge to properly build your package automatically. +::: + + + + +You can follow the same steps to setup your package on the real PyPI. However +in this lesson we will use testPyPI as a practice run. + +in xx lesson, you will learn how to setup an automated release workflow on GitHub +using GitHub actions that will automate the PyPI publication process whenever +you create a new software release. + +:::{admonition} Learning Objectives +:class: tip + +In this lesson you will learn how to: + +- how to build your package's sdist and wheel distributions +- Setup an account on testPyPI (the process is similar for the real PyPI) +- Publish your package to PyPI + +Once your package is on PyPI you can then easily publish it to conda-forge +using the [grayskull](https://conda.github.io/grayskull/) tool. You do not need to build the package specifically +for conda, conda-forge will build from your PyPI source distribution file (sdist). + +::: diff --git a/tutorials/image.png b/tutorials/image.png new file mode 100644 index 00000000..4d883066 Binary files /dev/null and b/tutorials/image.png differ diff --git a/tutorials/intro.md b/tutorials/intro.md index 1a9b9804..b569df00 100644 --- a/tutorials/intro.md +++ b/tutorials/intro.md @@ -1,4 +1,6 @@ # Python packaging 101 +*A start to finish beginner-friendly tutorial* + :::{admonition} Lesson Outline :class: important Below are the lessons contained in the packaging 101 series. In this series you will learn about the core elements that you need to publish your package to PyPI. - * What is a Python package? * [Make your code pip installable](1-installable-code.md) * Add a README file @@ -43,12 +51,15 @@ In the second series, you will learn about infrastructure and documentation need ```{toctree} :hidden: :caption: Python Packaging 101 -:glob: What is a Python package? Make your code pip installable <1-installable-code> +Add README file <2-add-readme> +Add LICENSE & CODE of CONDUCT <4-add-license-file> +Add pyproject.toml <5-pyproject-toml> +Publish to PyPI <6-publish-pypi> +Publish to conda-forge <7-publish-conda-forge> -* ``` ## What is a Python package? @@ -74,7 +85,7 @@ called. Diagram showing ADD ALT. -You can think about a package as a set of tools. A tool may be a function or a +You can think about a package as a toolbox filled with coding tools. A tool may be a function or a class. Each tool does a specific thing well. ::: @@ -98,7 +109,7 @@ TODOs: Diagram showing .. more here if this stays. -more here - might turn this into something more linear? +The elements of a Python package include code, documentation, tests a OSI-approved license and a specific directory structure. Maintainers are at the core making sure everything works and is up to date while fixing bugs and addressing user concerns. ::: The core elements of Python package include: @@ -112,11 +123,37 @@ The core elements of Python package include: If you intend for others to use and contribute to your code, consider who will maintain it over time. You will want a **contributing / development** guide to help new potential contributors get started with contributing to your package. And a **code of conduct** to ensure community interactions remain healthy both for you and your contributors / maintainer team -_Link to our EiC checklist for core components that pyOpenSci looks for… _ +:::{admonition} What pyOpenSci looks for in a package +:class: tip + +pyOpenSci will perform a set of checks for any package submitted. +You may find these checks useful as you create your package. +[Core components that pyOpenSci looks for.](https://www.pyopensci.org/software-peer-review/how-to/editor-in-chief-guide.html#editor-checklist-template) +::: + +## Packages are more than just code + +A package in any language is more than just code. If your package is public facing, meaning people besides yourself will use it, you should consider the various elements of a package that make it a useful community resource. + +### Version control and storing your package on GitHub or GitLab -A package in any language is more than just code. If your package is public facing, meaning people besides yourself will use it, you will want to think about the various elements of a package that make it a useful community resource. +Most Python packages live in an online version control platform such as GitHub or GitLab. GitHub and GitLab are both online platforms that +run [git](https://git-scm.com/) for version control. Have your software +under version control is important because it allows you to both track changes over time while also going back in history and undoing changes in the case that a change to the code base unexpectedly breaks something. -Most Python packages live in an online version control platform such as GitHub or GitLab. These platforms support robust infrastructure including continuous integration and continuous deployment (CI/CD). +By publishing your package on GitHub or GitLab, you are now making your code public facing. this means that others can both see your code and also make contributions using a pull request - review workflow. + +### Continuous integration and continuous deployment help you maintain your package + +Platforms such as GitHub and GitLab also provide continuous integration and continuous deployment (CI/CD). Continuous integration (CI) refers to a platform that automatically runs a specific job when a certain event occurs. + +**An example of Continuous integration:** +* When someone submits a change to your code, your tests will run across different operating systems and the code will be checked for format issues. + +**An example of Continuous deployment:** +* When you are ready to release your package to PyPI, a continuous deployment operation might be triggered on release to publish your package to PyPI. + +Integrated CI/CD will help you maintain your software ensuing that changes to the code don't break things unexpectedly and also maintain a style and format consistency. :::{figure-md} packaging-workflow @@ -126,13 +163,10 @@ The parts of a Python package... can this diagram be simplified a bit or at leas use simpler language?? maybe it's like code, docs, metadata, tests, all packaged up in a specific structure that tools can understand and turn into an installable package. that simpler version might be better. ::: ->then in the publish section below we can add the arrow with that structure being ->built into distributions that get published on pypi and conda... - ->then in the community part we add another section that is the github repo with users contributing... very simple diagrams... - - + ```{toctree} @@ -147,18 +181,28 @@ extras/* ## What should code in a Python package look like? Ideally the code in your Python package is general. This means it -can be applied in different settings. - -An example of a package that is written in a generalized way is `matplotlib`. +can be applied in different settings. An example of a package that is written in a generalized way is `matplotlib`. `matplotlib` does one (big important) thing really well: -It creates visual plots of data. +*It creates visual plots of data.* `Matplotlib` is used by thousands of users for different plotting applications using different types of data. While few scientific packages will have the same broad application as tools like `matplotlib` or `numpy`, the idea of code being used for something more than a single workflow still applies. +### Code should also be clean & readable & documented + +The code in your package should also be clean, readable and well documented. + +**Clean code:** Clean code refers to code that uses expressive variable names, is concise. + + +**Readable code:** Readable code is code written with a consistent style. You can use linters and code formatters such as black and flake8 to ensure this consistency throughout your entire package. [Learn more about code formatters here.](../package-structure-code/code-style-linting-format.html) + +**Documented code:** documented code is written using docstrings that help a user understand both what the functions and methods in your code does and also what the input and output elements of each function is. [You can learn more about docstrings in our guide, here.](../documentation/write-user-documentation/document-your-code-api-docstrings) + + :::{admonition} Where do research compendia fit in? :class: note @@ -174,6 +218,8 @@ which supports a specific project. ::: +## Making your package installable - publishing to PyPI & conda-forge + ### Python packages and environments A Python package can be installed into a Python environment in the same way @@ -187,17 +233,15 @@ allows you to access it from any code run with that specific Python environment This is just a concept - the graphic should show that you can turn your script into something that can be installed into a Python environment. Or you can turn your code into a package... ::: -## Publishing a package +### Publishing a package -If you want to make your package installable using `pip` you will want to +If you want to make your package installable using `pip` you need to publish it in a repository such as PyPI or conda-forge. If you package is a -pure python package, then luckily publish to both takes just a few steps. +pure python package, then publishing to both PyPI and conda-forge only takes just a few steps. -First, you will learn how to publish to PyPI using twine. -Then you can create a conda-forge recipe using the grayskull tool to easily publish to conda-forge from PyPI. you will learn more about the conda-forge publication process here. *TODO add link when lesson is created * +In these tutorials, you will learn [how to publish to PyPI using twine.](6-publish-pypi.md) -You will learn how -to publish to both [PyPI](4-publish-pypi.md) and conda forge in this tutorial series. +Then you can create a conda-forge recipe using the grayskull tool. This recipe can then be submitted to conda-forge. [You will learn more about the conda-forge publication process here.](7-publish-conda-forge.md) :::{figure-md} publish-package @@ -260,3 +304,9 @@ _TODO: I suspect nick murphy has something in this space that we can rif off of. In future lessons we will talk more about the infrastructure around a published Python package that makes it both easier to maintain, easier for others to contribute to and easier for other scientists to use. However, first we want to get you to your initial goal of publishing a Python package. In this next lesson you will learn how to create a basic installable Python package. + + + + +