You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 90e7d6e
Browse filesBrowse the repository at this point in the historyBrowse files
Include array shape in generated annotations (#132)
This bundles a few things that I should have probably committed separately while
iterating on them:
1. Closes#76. Shape information in doctypes is now translated into annotations. This
even goes for shapes with optional dimensions – which use expansion like in the
plugin example to create a union of arrays – one for each supported dimensionality.
Shapes with "..." are treated as unbounded dimensionality and translated to
`tuple[Any, ...]`
like [NumPy v2.5 suggests](https://numpy.org/doc/stable/reference/typing.html#api).
Empty shapes are supported too.
3. Dtype unions are now also expanded into a union of arrays each with one of the
dtypes. This is because I'm not sure if `np.dtype` actually supports unions. I can't
find examples of that in NumPy's repo. For now, expansion should be on the safe side.
4. Closes#125. Shape and dtype are now stripped from *ArrayLikes* which don't support
them as their are a union that can't be subscribed. This means, *ArrayLikes* are a
really generic broad type.
5. Cleans up some issues with the grammar: the literal and subscriptions rules have some
necessary overlap. The subscription rule should now win unless the expression
contains an actual literal type.
Also some refactoring, and especially new stuff like `expand_expression`,
`Expr.{find_all|try_find_one|find_one}` enable all this.
Docstub defines its own [grammar](../src/docstub/doctype.lark) to parse and transform type information in docstrings into valid Python type expressions.
4
-
This grammar fully supports [Python's conventional typing syntax](https://typing.python.org/en/latest/index.html).
5
-
So any {term}`annotation expression` that is valid in Python, can be used in a docstrings as is.
6
-
In addition, docstub extends this syntax with several "natural language" expressions that are commonly used in the scientific Python ecosystem.
3
+
Any valid Python {term}`annotation expression` can be used in a docstrings as is.
4
+
However, Docstub extends [Python's conventional typing syntax](https://typing.python.org/en/latest/index.html) with several "natural language" expressions that are commonly used in the scientific Python ecosystem.
5
+
This extended syntax is defined in its own [grammar](#appendix-grammar).
7
6
8
7
Docstrings should follow a form that is inspired by the [NumPyDoc style](https://numpydoc.readthedocs.io/en/latest/format.html):
9
8
```none
@@ -28,83 +27,87 @@ Combining multiple names that share a doctype and description is supported.
28
27
29
28
In addition to Python's conventional shorthand `|` syntax for [union types](https://typing.python.org/en/latest/spec/concepts.html#union-types), you can use `or` to join types.
30
29
31
-
|Docstring type | Python type annotation |
32
-
|----------------|------------------------|
33
-
|`X or Y`|`X \| Y`|
34
-
|`int or float`|`int \| float`|
30
+
|doctype | Python annotation |
31
+
|----------------|-------------------|
32
+
|`X or Y`|`X \| Y`|
33
+
|`int or float`|`int \| float`|
35
34
36
35
37
36
## Containers
38
37
39
38
The content of containers can be typed using a `CONTAINER of X` like form.
40
39
This extends the basic subscription syntax for [generics](https://typing.python.org/en/latest/spec/generics.html#generics).
For the simple case `CONTAINER of X`, where `X` is a name, you can append `(s)` to indicate the plural form.
48
47
For example, `list of float(s)`.
49
48
50
49
Variants of for [**tuples**](https://typing.python.org/en/latest/spec/tuples.html)
51
50
52
-
|Docstring type| Python type annotation |
53
-
|---------------------|------------------------|
54
-
|`tuple of (X, Y)`|`tuple[X, Y]`|
55
-
|`tuple of (X, ...)`|`tuple[X, ...]`|
51
+
|doctype | Python annotation |
52
+
|---------------------|-------------------|
53
+
|`tuple of (X, Y)`|`tuple[X, Y]`|
54
+
|`tuple of (X, ...)`|`tuple[X, ...]`|
56
55
57
56
and **mappings** exist.
58
57
59
-
|Docstring type| Python type annotation |
60
-
|----------------------|------------------------|
61
-
|`MAPPING of {X: Y}`|`MAPPING[X, Y]`|
62
-
|`dict of {str: int}`|`dict[str, int]`|
58
+
|doctype | Python annotation |
59
+
|----------------------|-------------------|
60
+
|`MAPPING of {X: Y}`|`MAPPING[X, Y]`|
61
+
|`dict of {str: int}`|`dict[str, int]`|
63
62
64
63
65
64
:::{tip}
66
-
While it is possible to nest these variants repeatedly, it decreases the readability.
65
+
While it is possible to nest these variants, it decreases the readability.
67
66
For complex nested annotations with nested containers, consider using Python's conventional syntax.
68
67
In the future, docstub may warn against or disallow nesting these natural language variants.
69
68
:::
70
69
71
70
72
71
(natlang-array)=
73
-
## Shape and dtype syntax for arrays
72
+
## NDArrays with shape and dtype
74
73
75
-
This expression allows adding shape and datatype information for data structures like [NumPy arrays](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html).
74
+
This "natural language" expression describes shape and dtype information for [NumPy arrays](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html).
75
+
The basic form is as follows:
76
76
77
-
`array` and `ndarray`, and `array-like` and `array_like` can be used interchange-ably for the variable `ARRAY` below.
|`array of dtype int`|`NDArray[tuple[Any, ...], np.dtype[int]]`|
98
+
|`array of shape (..., 2)`|`NDArray[tuple[Any, ...]]`|
99
+
|`array of shape (x, y[, z])`|`NDArray[tuple[int, int]]`<br>`\| NDArray[tuple[int, int, int]]`|
100
+
|`ndarray of dtype bool and shape (4, 4)`|`NDArray[tuple[int, int], np.dtype[bool]]`|
101
+
|`array-like of dtype float`|`ArrayLike`|
102
+
|`array_like of shape (M, 2)`|`ArrayLike`|
100
103
101
104
102
105
## Literals
103
106
104
107
[Literals](https://typing.python.org/en/latest/spec/literal.html#literals) indicate a concrete value instead of type.
105
108
Instead of using [`typing.Literal`](https://docs.python.org/3/library/typing.html#typing.Literal), you can enclose literal values in `{...}` in docstrings.
@@ -117,19 +120,27 @@ However, `Literal[X]` is more explicit.
117
120
:::{warning}
118
121
Python's `typing.Literal` only supports a restricted set of parameters.
119
122
For example, `float` literals are not yet supported by the type system but are allowed by docstub.
120
-
Addressing this use case is on the roadmap.
121
-
See [issue 47](https://github.com/scientific-python/docstub/issues/47) for more details.
123
+
Use a broader inlined type such as `Literal[0, 1] | float` if you want to use something like `{0, .5, 1}` in the docstring.
122
124
:::
123
125
124
126
## reStructuredText role
125
127
126
128
Since docstrings are also used to generate documentation with Sphinx, you may want to use [restructuredText roles](https://docutils.sourceforge.io/docs/ref/rst/roles.html).
127
-
This is supported anywhere in where a {term}`type name` is used in a {term}`doctype`.
0 commit comments