Get inclusion of markdown files in the User's Gudie to properly render

[ekluzek]

Brief summary of bug

The markdown files that are included in the User's Guide are rendering as literal raw text rather than as Myst Markdown files. For some this is because of the use of the "literal" keyword. But, in other places doc-builder needs to be updated for it to render correctly. The build step will also need to have a conda environment that has the myst_parser, so this probably needs to be added to ctsm_pylib.

General bug information

CTSM version you are using: ctsm5.4.035

Does this bug cause significantly incorrect results in the model's science? No

Configurations affected: Documentation build

Details of bug

The problem is described here:

ESMCI/doc-builder#42

Important details of your setup / configuration so we can reproduce the bug

Use of literal are shown here:

git grep :literal:
overview/introduction.rst:   :literal:
testing/testing.rst:   :literal:

If myst_parser needs to be added to ctsm_pylib that goes in the python directory.

Important output or errors that show the problem

See for example:

https://escomp.github.io/CTSM/users_guide/overview/introduction.html#readme-file-describing-version

Definition of Done:

• Update to doc-builder version with the main fix
• Fix the literal instances
• Add mys_parser to ctsm_pylib?
• Make sure with the changes the markdown files render correctly
• Merge PR with changes to b4b-dev

Pinging @samsrabin

[samsrabin]

@ekluzek Can you show me examples of where this problem is occurring without literal?

[samsrabin]

And what exactly is the problem requiring "Fix the literal instances"? It sounds like they are CORRECTLY rendering literally (i.e., as code-looking blocks rather than formatted text).

[samsrabin]

I've just looked through the docs and I see no problems. All instances of Markdown files being .. include::d in reStructuredText files are marked as either :literal: or :code: markdown.

[ekluzek]

Here's an example of the Quickstart.

https://escomp.github.io/CTSM/users_guide/overview/quickstart.html

You have a good point about the use of "literal". This might be the right thing in some cases. But, I also think I used it in order to get the content displayed before we added myst_parser to the build. So I think we should look at those two cases and decide if we really want the use of ":literal:" there. I don't think we should in the introduction, but possibly should in the unit-testing section. I think in most cases we want the markdown to render -- although we also have to realize that github markdown is slightly different from Myst, so that might come into play here.

[ekluzek]

@slevis-lmwg and I noticed this in the review of his work on mksurfdata_esmf in #3199. We realized it was outside the scope of his PR, but was something that should be addressed.

[samsrabin]

The Quickstart is included as literal code with Markdown syntax highlighting:

CTSM/doc/source/users_guide/overview/quickstart.rst

Lines 28 to 31 in e1f563b

	The Quickstart.GUIDE (which can be found in ``$CTSMROOT/doc``) is repeated here.
	.. include:: ../../../Quickstart.GUIDE.md
	:code: markdown

myst_parser is already included in the doc-building container and therefore ctsm_pylib (due to its environment file including the container's requirements.txt). It's also already included in doc-builder's conf.py.

As far as I know, there's no way to embed Markdown in a reStructuredText file. The only solution would be to convert the reStructuredText file to Markdown and then embed as described here.

@ekluzek and I just spoke, and he'll open a separate issue about doing that.