Release/2.5.0 (ramonhagenaars#101)

* Made the help text for functions with nptyping type hints become more helpful. * Updated some constraints. * Made the wheel test run only on 1 Python version (no differences expected between versions). * Updated tasks. * Allowed generic wildcard in structure expressions to allow "any columns". * Updated year in copyright statement * Attempt at enabling mypy-dataframe-tests on 3.11 on Actions. * Patch for invoke for python3.11 * Added note. * Added python3.12 to Actions * Woops * Update readme * Updated docs with global wildcard * Updated the packaging and distribution. * Version bump. * Documentation * Updated Readme. * Updated History. --------- Co-authored-by: Ramon <p8u7wAPC5Pg9HYkkCkzA>
jasper-tms · Feb 20, 2023 · 085487d · 085487d
1 parent 6f83693
commit 085487d
Show file tree

Hide file tree

Showing 37 changed files with 298 additions and 99 deletions.
diff --git a/.github/workflows/pythonapp.yml b/.github/workflows/pythonapp.yml
@@ -8,8 +8,7 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        python-version: [ '3.7', '3.8', '3.9', '3.10' ]
-        # FIXME: 3.11 fails because of invoke. See https://github.com/pyinvoke/invoke/pull/877
+        python-version: [ '3.7', '3.8', '3.9', '3.10', '3.11' ]
         os: [ ubuntu-latest, macOS-latest, windows-latest ]
     name: Python ${{ matrix.python-version }} on ${{ matrix.os }}
     steps:

diff --git a/HISTORY.md b/HISTORY.md
@@ -1,5 +1,12 @@
 # History
 
+## 2.5.0 (2023-02-20)
+
+- Added the column wildcard in structure expressions to allow expressing 'a structure with at least ...'. 
+- Fixed the `help` text for functions that use `nptyping` types as hints.
+- Fixed the distribution of `dataframe.pyi` that was missing.
+- Fixed the sdist to include tests and dependencies.
+
 ## 2.4.1 (2022-11-16)
 
 - Fixed compatibility with `mypy==0.991`.

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -0,0 +1,12 @@
+# See also:
+# https://packaging.python.org/guides/using-manifest-in/#how-files-are-included-in-an-sdist
+include CONTRIBUTING.md
+include HISTORY.md
+include USERDOCS.md
+include dependencies/*
+include dependencies/**/*
+include resources/*
+include resources/**/*
+include tests/*.py
+include tests/**/*.py
+recursive-exclude *.pyc
diff --git a/README.md b/README.md
@@ -11,9 +11,10 @@
   </a> 
 </p>
 
-💡 *Type hints for `NumPy`* <br/>
-💡 *Type hints for `pandas.DataFrame`* <br/>
+🧊 *Type hints for `NumPy`* <br/>
+🐼 *Type hints for `pandas.DataFrame`* <br/>
 💡 *Extensive dynamic type checks for dtypes shapes and structures* <br/>
+🚀 *[Jump to the Quickstart](https://github.com/ramonhagenaars/nptyping/blob/master/USERDOCS.md#Quickstart)*
 
 Example of a hinted `numpy.ndarray`:
 
@@ -32,15 +33,14 @@ Example of a hinted `pandas.DataFrame`:
 >>> df: DataFrame[S["name: Str, x: Float, y: Float"]]
 
 ```
-⚠️`pandas.DataFrame` is not yet supported on Python 3.11.
 
 ### Installation
 
-| Command                          | Description                                               |
-|:---------------------------------|-----------------------------------------------------------|
-| `pip install nptyping`           | Install the basics                                        |
-| `pip install nptyping[pandas]`   | Install with pandas extension (⚠️Python 3.10 or lower)  |
-| `pip install nptyping[complete]` | Install with all extensions                               |
+| Command                          | Description                   |
+|:---------------------------------|-------------------------------|
+| `pip install nptyping`           | Install the basics            |
+| `pip install nptyping[pandas]`   | Install with pandas extension |
+| `pip install nptyping[complete]` | Install with all extensions   |
 
 ### Instance checking
 

diff --git a/USERDOCS.md b/USERDOCS.md
@@ -5,21 +5,23 @@
 # *User documentation*
 
 * [Introduction](#Introduction)
+* [Quickstart](#Quickstart)
 * [Usage](#Usage)
     * [NDArray](#NDArray)
     * [Shape expressions](#Shape-expressions)
         * [Syntax](#Syntax-shape-expressions)
         * [Validation](#Validation)
         * [Normalization](#Normalization)
         * [Variables](#Variables)
-        * [Wildcards](#Wildcards)
+        * [Wildcards](#Shape-Wildcards)
         * [N dimensions](#N-dimensions)
         * [Dimension breakdowns](#Dimension-breakdowns)
         * [Labels](#Labels)
     * [DTypes](#DTypes)
     * [Structure expressions](#Structure-expressions)
       * [Syntax](#Syntax-structure-expressions)
       * [Subarrays](#Subarrays)
+      * [Wildcards](#Structure-Wildcards)
     * [RecArray](#RecArray)
     * [Pandas DataFrame](#Pandas-DataFrame)
 * [Examples](#Examples)
@@ -43,6 +45,32 @@ raise your question [in a new issue](https://github.com/ramonhagenaars/nptyping/
 You will find a lot of code blocks in this document. If you wonder why they are written the way they are (e.g. with the 
 `>>>` and the `...`): all code blocks are tested using [doctest](https://docs.python.org/3/library/doctest.html).
 
+## Quickstart
+Install `nptyping` for the type hints and the recommended `beartype` for dynamic type checks:
+```shell
+pip install nptyping[complete], beartype
+```
+
+Use the combination of these packages to add type safety and readability:
+```python
+# File: myfile.py
+
+>>> from nptyping import DataFrame, Structure as S
+>>> from beartype import beartype
+
+>>> @beartype  # The function signature is now type safe
+... def fun(df: DataFrame[S["a: int, b: str"]]) -> DataFrame[S["a: int, b: str"]]:
+...     return df
+
+```
+
+On your production environments, run Python in optimized mode. This disables the type checks done by beartype and any
+overhead it may cause:
+```shell
+python -OO myfile.py
+```
+You're now good to go. You can sleep tight knowing that today you made your codebase safer and more transparent.
+
 ## Usage
 
 ### NDArray
@@ -173,7 +201,7 @@ They are interpreted from left to right. This means that in the last example, up
 
 A variable is a word that may contain underscores and digits as long as *it starts with an uppercase letter*.
 
-#### Wildcards
+#### Shape Wildcards
 A wildcard accepts any dimension size. It is denoted by the asterisk (`*`). Example:
 ```python
 >>> isinstance(random.randn(42, 43), NDArray[Shape["*, *"], Any])
@@ -331,7 +359,7 @@ NDArray[Any, Floating]
 
 ### Structure expressions
 You can denote the structure of a structured array using what we call a **structure expression**. This expression 
-- again a string - can be put into `Structure` and can then be used in an `NDArray`.
+(again a string) can be put into `Structure` and can then be used in an `NDArray`.
 ```python
 >>> from nptyping import Structure
 
@@ -389,7 +417,7 @@ True
 The syntax of a structure expression can be formalized in BNF. Extra whitespacing is allowed (e.g. around commas and 
 colons), but this is not included in the schema below.
 ```
-structure-expression  =  <fields>
+structure-expression  =  <fields>|<fields>","<wildcard>
 fields                =  <field>|<field>","<fields>
 field                 =  <field-name>":"<field-type>|"["<combined-field-names>"]:"<field-type>
 combined-field-names  =  <field-name>","<field-name>|<field-name>","<combined-field-names>
@@ -420,6 +448,23 @@ True
 
 ```
 
+#### Structure Wildcards
+You can use wildcards for field types or globally (for complete fields).
+Here is an example of a wildcard for a field type:
+```python
+>>> Structure["anyType: *"]
+Structure['anyType: *']
+
+```
+
+And here is an example with a global wildcard:
+```python
+>>> Structure["someType: int, *"]
+Structure['someType: int, *']
+
+```
+This expresses a structure that has *at least* a field `someType: int`. Any other fields are also accepted.
+
 ### RecArray
 The `RecArray` corresponds to [numpy.recarray](https://numpy.org/doc/stable/reference/generated/numpy.recarray.html).
 It is an extension of `NDArray` and behaves similarly. A key difference is that with `RecArray`, the `Structure` OR 

diff --git a/constraints-3.10.txt b/constraints-3.10.txt
@@ -49,9 +49,9 @@ lazy-object-proxy==1.7.1
     # via astroid
 mccabe==0.7.0
     # via pylint
-mypy==0.991
+mypy==1.0.0
     # via -r ./dependencies\qa-requirements.txt
-mypy-extensions==0.4.3
+mypy-extensions==1.0.0
     # via
     #   black
     #   mypy
@@ -81,7 +81,7 @@ pylint==2.15.2
     # via -r ./dependencies\qa-requirements.txt
 pyparsing==3.0.9
     # via packaging
-pyright==1.1.273
+pyright==1.1.294
     # via -r ./dependencies\qa-requirements.txt
 python-dateutil==2.8.2
     # via pandas

diff --git a/constraints-3.11.txt b/constraints-3.11.txt
@@ -49,9 +49,9 @@ lazy-object-proxy==1.8.0
     # via astroid
 mccabe==0.7.0
     # via pylint
-mypy==0.991
+mypy==1.0.0
     # via -r ./dependencies\qa-requirements.txt
-mypy-extensions==0.4.3
+mypy-extensions==1.0.0
     # via
     #   black
     #   mypy
@@ -81,7 +81,7 @@ pylint==2.15.5
     # via -r ./dependencies\qa-requirements.txt
 pyparsing==3.0.9
     # via packaging
-pyright==1.1.279
+pyright==1.1.294
     # via -r ./dependencies\qa-requirements.txt
 python-dateutil==2.8.2
     # via pandas

diff --git a/constraints-3.7.txt b/constraints-3.7.txt
@@ -56,7 +56,7 @@ mccabe==0.7.0
     # via pylint
 mypy==0.991
     # via -r ./dependencies\qa-requirements.txt
-mypy-extensions==0.4.3
+mypy-extensions==1.0.0
     # via
     #   black
     #   mypy
@@ -86,7 +86,7 @@ pylint==2.15.2
     # via -r ./dependencies\qa-requirements.txt
 pyparsing==3.0.9
     # via packaging
-pyright==1.1.273
+pyright==1.1.294
     # via -r ./dependencies\qa-requirements.txt
 python-dateutil==2.8.2
     # via pandas

diff --git a/constraints-3.8.txt b/constraints-3.8.txt
@@ -49,9 +49,9 @@ lazy-object-proxy==1.7.1
     # via astroid
 mccabe==0.7.0
     # via pylint
-mypy==0.991
+mypy==1.0.0
     # via -r ./dependencies\qa-requirements.txt
-mypy-extensions==0.4.3
+mypy-extensions==1.0.0
     # via
     #   black
     #   mypy
@@ -81,7 +81,7 @@ pylint==2.15.2
     # via -r ./dependencies\qa-requirements.txt
 pyparsing==3.0.9
     # via packaging
-pyright==1.1.273
+pyright==1.1.294
     # via -r ./dependencies\qa-requirements.txt
 python-dateutil==2.8.2
     # via pandas

diff --git a/constraints-3.9.txt b/constraints-3.9.txt
@@ -49,9 +49,9 @@ lazy-object-proxy==1.7.1
     # via astroid
 mccabe==0.7.0
     # via pylint
-mypy==0.991
+mypy==1.0.0
     # via -r ./dependencies\qa-requirements.txt
-mypy-extensions==0.4.3
+mypy-extensions==1.0.0
     # via
     #   black
     #   mypy
@@ -81,7 +81,7 @@ pylint==2.15.2
     # via -r ./dependencies\qa-requirements.txt
 pyparsing==3.0.9
     # via packaging
-pyright==1.1.273
+pyright==1.1.294
     # via -r ./dependencies\qa-requirements.txt
 python-dateutil==2.8.2
     # via pandas

diff --git a/nptyping/__init__.py b/nptyping/__init__.py
@@ -1,7 +1,7 @@
 """
 MIT License
 
-Copyright (c) 2022 Ramon Hagenaars
+Copyright (c) 2023 Ramon Hagenaars
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

diff --git a/nptyping/assert_isinstance.py b/nptyping/assert_isinstance.py
@@ -1,7 +1,7 @@
 """
 MIT License
 
-Copyright (c) 2022 Ramon Hagenaars
+Copyright (c) 2023 Ramon Hagenaars
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

diff --git a/nptyping/base_meta_classes.py b/nptyping/base_meta_classes.py
@@ -1,7 +1,7 @@
 """
 MIT License
 
-Copyright (c) 2022 Ramon Hagenaars
+Copyright (c) 2023 Ramon Hagenaars
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -22,9 +22,11 @@
 SOFTWARE.
 """
 from abc import ABCMeta, abstractmethod
+from inspect import FrameInfo
 from typing import (
     Any,
     Dict,
+    List,
     Optional,
     Set,
     Tuple,
@@ -123,6 +125,11 @@ class SubscriptableMeta(ABCMeta):
     def _get_item(cls, item: Any) -> Tuple[Any, ...]:
         ...  # pragma: no cover
 
+    def _get_module(cls, stack: List[FrameInfo], module: str) -> str:
+        # The magic below makes Python's help function display a meaningful
+        # text with nptyping types.
+        return "typing" if stack[1][3] == "formatannotation" else module
+
     def _get_additional_values(
         cls, item: Any  # pylint: disable=unused-argument
     ) -> Dict[str, Any]:

diff --git a/nptyping/error.py b/nptyping/error.py
@@ -1,7 +1,7 @@
 """
 MIT License
 
-Copyright (c) 2022 Ramon Hagenaars
+Copyright (c) 2023 Ramon Hagenaars
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

diff --git a/nptyping/ndarray.py b/nptyping/ndarray.py
@@ -1,7 +1,7 @@
 """
 MIT License
 
-Copyright (c) 2022 Ramon Hagenaars
+Copyright (c) 2023 Ramon Hagenaars
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -21,6 +21,7 @@
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
 """
+import inspect
 from abc import ABC
 from typing import Any, Tuple
 
@@ -64,6 +65,10 @@ class NDArrayMeta(
     __args__: Tuple[Shape, DType]
     _parameterized: bool
 
+    @property
+    def __module__(cls) -> str:
+        return cls._get_module(inspect.stack(), "nptyping.ndarray")
+
     def _get_item(cls, item: Any) -> Tuple[Any, ...]:
         cls._check_item(item)
         shape, dtype = cls._get_from_tuple(item)

diff --git a/nptyping/ndarray.pyi b/nptyping/ndarray.pyi
@@ -1,7 +1,7 @@
 """
 MIT License
 
-Copyright (c) 2022 Ramon Hagenaars
+Copyright (c) 2023 Ramon Hagenaars
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

diff --git a/nptyping/nptyping_type.py b/nptyping/nptyping_type.py
@@ -1,7 +1,7 @@
 """
 MIT License
 
-Copyright (c) 2022 Ramon Hagenaars
+Copyright (c) 2023 Ramon Hagenaars
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal