feat!: enable type checking of funparam function arguments

To enable type checking on funparam function arguments, I need to change
the API. Currently, we're using "magic" keyword arguments: `_id` and
`_marks`. There's no way to support this with Python's current static
typing.

The best way I could come up with was making funparam functions a class,
where the `__call__` method has the type signature of the wrapped
function. The funparam function can accept an `id` or `marks` via its
`.id()` and `.marks()` methods. Both methods return a funparam function
object with the same signature as the originally wrapped function.

It's kind of gross-looking, but it gets the job done. I'll keep
exploring alternatives. For now, this is an improvement.

This commit also implements some testing utilities for validating static
typing behavior with mypy.

BREAKING CHANGE: Using the `_marks` and `_id` keyword arguments is no
longer supported.

QUIRKS.rst

@@ -122,7 +122,7 @@ within another fixture called ``funparam``:
     >       available fixtures: *
     >       use 'pytest --fixtures [testpath]' for help on them.
 
-    */pytest_funparam/__init__.py:266
+    */pytest_funparam/__init__.py:*
     =========================== short test summary info ============================
     ERROR test_quirks.py::test_addition
     =============================== 1 error in 0.02s ===============================

README.rst

@@ -4,17 +4,25 @@ pytest-funparam
 
 ``pytest-funparam`` makes it easy to write parametrized tests.
 
+Unlike ``pytest.mark.parametrize``, ``pytest-funparam``:
+
+* includes the failing parameter in pytest tracebacks;
+* enables static type checking of parameters; and
+* keeps parameters and assertions closer together.
+
+.. contents::
+
 
 Installation
-------------
+============
 
 You can install "pytest-funparam" via `pip`_ from `PyPI`_::
 
     $ pip install pytest-funparam
 
 
 Usage
------
+=====
 
 Inside a test function, decorate a function with the ``funparam`` fixture:
 
@@ -100,7 +108,10 @@ commands like ``pytest --last-failed``::
     ============================== 1 failed in 0.01s ===============================
 
 
-Mark tests by using the ``_marks`` keyword argument on calls to verify:
+Markers
+-------
+
+Mark tests by using the ``.marks()`` method of your funparam function.
 
 .. code-block:: python
 
@@ -112,7 +123,7 @@ Mark tests by using the ``_marks`` keyword argument on calls to verify:
             assert a + b == expected
 
         verify_sum(1, 2, 3)
-        verify_sum(2, 2, 5, _marks=pytest.mark.skip)
+        verify_sum.marks(pytest.mark.skip)(2, 2, 5)
         verify_sum(4, 2, 6)
 
 ::
@@ -126,11 +137,39 @@ Mark tests by using the ``_marks`` keyword argument on calls to verify:
     ========================= 2 passed, 1 skipped in 0.01s =========================
 
 
-Note that the ``_marks`` keyword argument is passed through directly to the
-``marks`` keyword argument of ``pytest.mark.param()``. This means the value can
-be either a single mark or a collection of marks.
+Test IDs
+--------
+
+Similarly, add an ``id`` to a test using the ``.id()`` method of your funparam
+function:
+
+.. code-block:: python
+
+    def test_addition(funparam):
+        @funparam
+        def verify_sum(a, b, expected):
+            assert a + b == expected
+
+        verify_sum.id("one and two")(1, 2, 3)
+        verify_sum.id("two and two")(2, 2, 5)
+        verify_sum.id("four and two")(4, 2, 6)
+
+::
+
+    $ pytest --collect-only
+    ============================= test session starts ==============================
+    collected 3 items
+
+    <Module test_readme.py>
+      <Function test_addition[one and two]>
+      <Function test_addition[two and two]>
+      <Function test_addition[four and two]>
+
+    ========================== 3 tests collected in 0.01s ==========================
+
 
-Similarly, add an ``id`` to a test using the ``_id`` keyword argument:
+You can also use the shorthand for assigning an ``id``. (It does the same thing
+as calling ``.id()``.)
 
 .. code-block:: python
 
@@ -139,9 +178,9 @@ Similarly, add an ``id`` to a test using the ``_id`` keyword argument:
         def verify_sum(a, b, expected):
             assert a + b == expected
 
-        verify_sum(1, 2, 3, _id="one and two")
-        verify_sum(2, 2, 5, _id="two and two")
-        verify_sum(4, 2, 6, _id="four and two")
+        verify_sum["one and two"](1, 2, 3)
+        verify_sum["two and two"](2, 2, 5)
+        verify_sum["four and two"](4, 2, 6)
 
 ::
 
@@ -156,14 +195,52 @@ Similarly, add an ``id`` to a test using the ``_id`` keyword argument:
 
     ========================== 3 tests collected in 0.01s ==========================
 
+
+Type Annotations
+----------------
+
+``pytest-funparam`` has full type annotations. The ``funparam`` fixture returns
+a ``FunparamFixture`` object. You can import it from ``pytest_funparam``:
+
+.. code-block:: python
+
+    import pytest
+    from pytest_funparam import FunparamFixture
+
+    def test_addition(funparam: FunparamFixture):
+
+        @funparam
+        def verify_sum(a: int, b: int , expected: int):
+            assert a + b == expected
+
+        # These are valid
+        verify_sum(1, 2, 3)
+        verify_sum['it accommodates ids'](2, 2, 4)
+        # Marks work too!
+        verify_sum.marks(pytest.mark.xfail)(2, 2, 9)
+
+        # This will be marked as invalid (since it's not an int)
+        verify_sum(1, '2', 3)
+
+        # Using id/marks will still preserve the function's typing.
+        verify_sum['should be an int'](1, 2, '3')
+
+::
+
+    $ mypy
+    test_readme.py:17: error: Argument 2 to "verify_sum" has incompatible type "str"; expected "int"
+    test_readme.py:20: error: Argument 3 to "verify_sum" has incompatible type "str"; expected "int"
+    Found 2 errors in 1 file (checked 1 source file)
+
+
 License
--------
+=======
 
 Distributed under the terms of the `MIT`_ license, "pytest-funparam" is free and open source software
 
 
 Issues
-------
+======
 
 If you encounter any problems, please `file an issue`_ along with a detailed description.
 

src/pytest_funparam/__init__.py

@@ -1,6 +1,6 @@
 import pytest
 from unittest.mock import MagicMock
-from functools import wraps
+from functools import update_wrapper, wraps
 from typing import (
     TYPE_CHECKING,
     Any,
@@ -12,9 +12,14 @@
     Collection,
     Callable,
     Optional,
+    TypeVar,
+    Generic,
 )
 
 
+F = TypeVar('F', bound=Callable[..., None])
+
+
 if TYPE_CHECKING:  # pragma: no cover
     from _pytest.python import Metafunc, FunctionDefinition
     from _pytest.fixtures import FixtureDef
@@ -141,6 +146,63 @@ class NestedFunparamError(Exception):
     pass
 
 
+class IdentifiedFunparamFunction(Generic[F]):
+
+    def __init__(
+        self,
+        function: F,
+        *,
+        id: Union[str, None] = None,
+        marks: Collection['MarkDecorator'] = (),
+    ) -> None:
+        self._function = function
+        self._id = id
+        self._marks = marks
+        update_wrapper(self, function)
+
+    __call__: F
+
+    def __call__(self, *args, **kwargs):  # type: ignore
+        return self._function(
+            *args,
+            _id=self._id,
+            _marks=self._marks,
+            **kwargs
+        )
+
+    def marks(
+        self,
+        *marks: 'MarkDecorator'
+    ) -> "IdentifiedFunparamFunction[F]":
+        all_marks = (*self._marks, *marks)
+        return type(self)(
+            self._function,
+            id=self._id,
+            marks=all_marks,
+        )
+
+
+class UnidentifiedFunparamFunction(IdentifiedFunparamFunction[F]):
+
+    def id(self, id_: str) -> "IdentifiedFunparamFunction[F]":
+        return IdentifiedFunparamFunction(
+            self._function,
+            id=id_,
+            marks=self._marks,
+        )
+
+    def __getitem__(self, id_: str) -> "IdentifiedFunparamFunction[F]":
+        return self.id(id_)
+
+    def marks(
+        self,
+        *marks: 'MarkDecorator'
+    ) -> "UnidentifiedFunparamFunction[F]":
+        # HACK: Superclass uses `type(self)` to get class. We don't need to do
+        #       anything special to get the type correct.
+        return super().marks(*marks)  # type: ignore
+
+
 class FunparamFixture:
     """
     The base API for the `funparam` fixture.
@@ -168,16 +230,17 @@ def _make_key(self, verify_function: Callable[..., None]) -> int:
 
     def __call__(
         self,
-        verify_function: Callable[..., None]
-    ) -> Callable[..., None]:
+        verify_function: F,
+    ) -> UnidentifiedFunparamFunction[F]:
+
         key = self._make_key(verify_function)
         self.verify_functions[key] = verify_function
 
         @wraps(verify_function)
         def funparam_wrapper(*args: Any, **kwargs: Any) -> None:
             return self.call_verify_function(key, *args, **kwargs)
 
-        return funparam_wrapper
+        return UnidentifiedFunparamFunction(funparam_wrapper)  # type: ignore
 
 
 class GenerateTestsFunparamFixture(FunparamFixture):

tests/conftest.py

@@ -1,4 +1,5 @@
 pytest_plugins = [
     'pytester',
     "tests.fixtures.verify_examples",
+    "tests.fixtures.type_checking",
 ]

tests/fixtures/test_type_checking.py

@@ -0,0 +1,49 @@
+import pytest
+from .type_checking import MypyLine, parse_mypy_line
+
+
+def test_parse_mypy_line():
+    line = '<string>:16: error: Argument 2 to "verify_sum" has incompatible type "str"; expected "int"  [arg-type]'  # noqa
+    parsed = MypyLine(
+        source='<string>',
+        lineno=16,
+        severity="error",
+        message='Argument 2 to "verify_sum" has incompatible type "str"; expected "int"',  # noqa
+        code='arg-type',
+    )
+    assert parse_mypy_line(line) == parsed
+
+
+def test_assert_mypy_error_codes(assert_mypy_error_codes):
+    # Happy path
+    assert_mypy_error_codes(
+        """
+        foo: int
+        foo = 3
+        foo = 'bar'  # [assignment]
+        """
+    )
+    # Expected failure did not happen
+    with pytest.raises(AssertionError):
+        assert_mypy_error_codes(
+            """
+            foo: int
+            foo = 3  # [assigment]
+            """
+        )
+    # Did not expect failure
+    with pytest.raises(AssertionError):
+        assert_mypy_error_codes(
+            """
+            foo: int
+            foo = 'foo'
+            """
+        )
+    # Wrong error expected
+    with pytest.raises(AssertionError):
+        assert_mypy_error_codes(
+            """
+            foo: int
+            foo = 'foo'  # [arg-type]
+            """
+        )

tests/fixtures/test_verify_examples.py

@@ -57,3 +57,27 @@ def verify_sum(a, b, expected):
     )
 
     verify_examples(text)
+
+
+def test_verify_examples_mypy(verify_examples):
+    verify_examples(dedent(
+        """\
+        Given an example:
+
+        .. code-block:: python
+
+            def verify_sum(a: int, b: int, expected: int) -> None:
+                assert a + b == expected
+
+            verify_sum(1, 2, 3)
+            verify_sum("a", "b", "ab")
+
+        ::
+
+            $ mypy
+            test_verify_examples_mypy.py:5: error: Argument 1 to "verify_sum" has incompatible type "str"; expected "int"
+            test_verify_examples_mypy.py:5: error: Argument 2 to "verify_sum" has incompatible type "str"; expected "int"
+            test_verify_examples_mypy.py:5: error: Argument 3 to "verify_sum" has incompatible type "str"; expected "int"
+            Found 3 errors in 1 file (checked 1 source file)
+        """  # noqa
+    ))