XML code-blocks in nxdl.xsd are not picked up in the documentation

[lukaspie]

https://github.com/nexusformat/definitions/blob/main/nxdl.xsd#L1314 defines a code-block like this

.. code-block:: xml
	:linenos:
	
	<dimensions rank="2">
		<dim index="1" value="nsurf"/>
		<dim index="2" value="nwl"/>
	</dimensions>

inside an xs:documentation element of the nxdl.xsd file. However, this does not properly converted into a code block in the docs, see https://manual.nexusformat.org/nxdl_desc.html#dim and the screenshot below.

I believe the problem is in https://github.com/nexusformat/definitions/blob/main/dev_tools/docs/xsd.py#L265, where only the textual part of the documentation is retrieved and the XML is ignored.

There are two possible solutions:

• Use XML escape sequences directly in the nxdl.xsd file, i.e. write

  <dimensions rank="2">
      <dim index="1" value="nsurf"/>
      <dim index="2" value="nwl"/>
  </dimensions>
  

  I am doing this in extends attribute to fields to clarify if a field in a specified NXdata is a AXISNAME or DATA #1533.
• modify the function in the dev_tools to allow XML parts

@woutdenolf any ideas? Git blame says this code was originally written by you, but could just be copied from somewhere else.

[woutdenolf]

I'm in the blame because of the major refactoring in #1133. @prjemian was the original author.

My 5 cents so far: we do XML escaping all over the docs and then we unescape in these two places: for field types

definitions/dev_tools/docs/xsd.py

Line 275 in a852e06

text = html.unescape(text)

and for NeXus class definitions

definitions/dev_tools/docs/nxdl.py

Line 273 in a852e06

text = html.unescape(text)

In your case it is field types and the string being unescaped

'Specify the parameters for each index of the ``dimensions``\nelement with a ``dim`` element.\nThe number of ``dim`` entries should be equal to\nthe ``rank`` of the array.\nFor example, these terms\ndescribe a 2-D array with lengths (``nsurf``, ``nwl``):\n\n.. code-block:: xml\n\t:linenos:\n\n\n\n\n\n\nThe ``value`` attribute is used by NXDL and also by\nthe NeXus data file validation tools to associate and coordinate the\nsame array length across multiple fields in a group.'

After unescaping with the code

import html

text = html.unescape(text)

we have this string

'Specify the parameters for each index of the ``dimensions``\nelement with a ``dim`` element.\nThe number of ``dim`` entries should be equal to\nthe ``rank`` of the array.\nFor example, these terms\ndescribe a 2-D array with lengths (``nsurf``, ``nwl``):\n\n.. code-block:: xml\n\t:linenos:\n\n\n\n\n\n\nThe ``value`` attribute is used by NXDL and also by\nthe NeXus data file validation tools to associate and coordinate the\nsame array length across multiple fields in a group.'

The generated RST file ./build/manual/source/nxdl_desc.rst has it

    Specify the parameters for each index of the ``dimensions``
    element with a ``dim`` element.
    The number of ``dim`` entries should be equal to
    the ``rank`` of the array.
    For example, these terms
    describe a 2-D array with lengths (``nsurf``, ``nwl``):
    
    .. code-block:: xml
    	:linenos:
    
    
    
    
    
    
    The ``value`` attribute is used by NXDL and also by
    the NeXus data file validation tools to associate and coordinate the
    same array length across multiple fields in a group.

[woutdenolf]

So the issue is that html.unescape(text) removes the code that has < characters.

If I understand correctly, < is an XML entity used to escape < inside XML text blocks to prevent XML parsing errors.

So using < in XML text blocks is a no go regardless.

In other words, your solution in #1533 is the only possible solution I think.

[lukaspie]

Thanks, this is also what I thought. Should we then simply change the other instances in the nxdl.xsd file where actual XML code is used in the docs to an escaped code sequence? I think there is actually only one instance of this (the one mentioned above).