pytest: doctest support for reStructuredText, myst-parser, etc.

[tony]

pytest's doctest support (source) is great, the doctest standard library module is also used in docutils, as seen via Doctest Blocks (doctest_block: docutils reference, on pydoc.dev)

Sphinx also supports it via sphinx.ext.doctest (source, tests).

However, pytest in itself doesn't parse reStructuredText (requires docutils) and maybe sphinx-doc, nor does it handle markdown, e.g. parsing with myst-parser via markdown-it.

I'm of the belief this is an itch we need to scratch: this can, in tandem with projects like sphinx-copybutton, give us excellent, runnable examples that also act as test coverage and documentation. It's a huge win.

I would like us to be able to do: pytest doc/ and run doctest directives as any normal test, fixtures and all. We're a ways away, but just generating some attention for this effort.

One current project is at: https://github.com/thisch/pytest-sphinx/

Regardless of what comes of this, I hope we get bullet proof support for pytest + doctest in reStructuredText and markdown via a solid plugin. Perhaps it can live in the pytest-dev org.

If you're facing a similar issue please consider studying the relevant APIs and contributing some time to pytest-sphinx (above) or letting us know any other relevant efforts underway to make this a reality.

[twmr]

I'm totally in favor of using a proper rst or markdown parser for parsing the doctests in the file formats supported by those parsers, instead of relying on the doctest parser in the pytest doctest module, which just uses a regex for finding the doctests.

@tony is working on a WIP implementation of a docutils (for rst) myst-parser (for markdown) based pytest doctest plugin in twmr/pytest-sphinx#38

I would like us to be able to do: pytest doc/ and run doctest directives as any normal test, fixtures and all. We're a ways away, but just generating some attention for this effort.

👍🏿

What is your opinion on adding a few optional dependencies to pytest and using them, if installed, for parsing rst and markdown files?

Regardless of what comes of this, I hope we get bullet proof support for pytest + doctest in reStructuredText and markdown via a solid plugin. Perhaps it can live in the pytest-dev org.

If no one objects, we should include this in pytest-core.

[RonnyPfannschmidt]

i would like to see this establish first, integrating the stdlib module vs havinga complete parsing system that integrates with the documentation and all its dependencies are different pairs of shoes

my impression is that this one should be a recommended plugin