Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: parent-child convention for installed packages #1011

Merged
merged 4 commits into from
Nov 29, 2023
Merged

Doc: parent-child convention for installed packages #1011

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
285e9c5
Doc: parent-child convention for installed packages
Julow Sep 28, 2023
132f52e
tentative convention for building installed package's doc
panglesd Oct 20, 2023
fdc1a75
Remove the "index" tentative fix
Julow Nov 10, 2023
5d0543c
convention: Remove paragraph on source rendering bug
Julow Nov 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions doc/parent_child_spec.mld
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
{0 Parent/Child Specification}
This parent/child specification allows more flexible output support, e.g., per library documentation. See {{:https://ocaml.org/packages}ocaml.org/packages}.

{1 Rules}

The rules are;

{ul
Expand All @@ -19,6 +21,8 @@ The rules are;
{b Note:} The [--pkg <package>] option is still supported for backward compatibility in [odoc >= v2.0.0],
although it's now equivalent to specifying a parent [.mld] file.

{1 Example}

For example, let's consider [John] whose is [Doe] and [Mark]'s father. [Doe] has
children, [Max], and page [foo], whereas [Mark] has no children. That is to say,
[john.mld], [doe.mld], [mark.mld], [max.mld], [foo.ml] respectively. For instance;
Expand Down Expand Up @@ -157,3 +161,34 @@ and we shall get
]}

For more about [odoc] commands, simply invoke [odoc --help] in your shell.

{1 Convention for installed packages}

Locally, the build system can make arbitrary complex documentation page
hierarchies.
However, the generated HTML documentation is generally not installed as part of
a package. Instead the documentation source code made of [.mld] pages is
installed and might be used by a different driver.

{2 Convention}

In order for drivers to build consistent documentation for a package, the
following convention should be followed.

- [.mld] pages are installed in a package's [share] directory, under the
[odoc-pages] sub-directory.
- [index.mld] is the parent of every other pages. The driver can freely rename
this page, for example, it can be named after the package.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The renaming bit doesn't work, it prevents from making links from other pages into the index page (example).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The index of package page is not generated using Odoc ? It can't make references to {!page-index}, as that would be very ambiguous.

Though I believe we need references through the page hierarchy like {!page-packages.odoc.index} but even then, the index part isn't wanted.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we'll want to allow deeper hierarchies in the future and our first though is to generalize the index.mld renaming:

If you have foo/bar/index.mld, it makes a page named foo/bar.html and have foo/bar/*.mld as children.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dbuenzli you're right, linking to the index is currently broken. We can fix this by having links to page-index resolve to the parent, but it all feels a little hacky. If anyone has any other bright ideas I'll be very happy to hear suggestions.

The index of package page is not generated using Odoc ? It can't make references to {!page-index}, as that would be very ambiguous.

It's only ambiguous knowing how the various drivers work - it's entirely unambiguous in the context of having a flat hierarchy of mld pages per package - ie, the view that the package author has. There can be only 1 index.mld file, as they're all stored in the same directory (and dune at least enforces this - I presume other packaging tools do similarly.) When we have support for user-written hierarchical docs, we can be more explicit in what to name your mlds, how the hierarchy works, how to make links between pages correctly and whatnot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The index of package page is not generated using Odoc ?

I'm not sure what you are asking here. The index of the package page is generated by processing the index.mld file provided by the package or one generated by the driver if none is provided.

Note that currently odig is actually using the legacy --pkg option (I'm planning to rewrite the driver any time soon). So I'm wondering whether currently odig is not maybe making something that is not described by this convention in this PR and we are missing a bit.

It can't make references to {!page-index}, as that would be very ambiguous.

I notice for the example I mention that while the link gets correctly rendered in the brzo driver which I use for docsets on erratique.ch, it's neither rendered by odig nor by ocaml.org.

Though I believe we need references through the page hierarchy like {!page-packages.odoc.index} but even then, the index part isn't wanted.

For cross packages links I don't really care if there's an additional .index needed. However one thing that is important is not to have to hard code your package/parent name everywhere. Even for your example of deeper hierarchies you want to be able to refer to your parent's index page without having to name it.

Besides I guess that if we want these cross packages links to work. Then we need to go a bit beyond what is described here, on how things need to be compiled. (In the legacy pipeline the --pkg option made all that clear).

If you have foo/bar/index.mld, it makes a page named foo/bar.html and have foo/bar/*.mld as children.

I don't really care how the page end up being named but we need a reliable way to be able to refer to these in the sources. However in general the idea of solving these problems by renaming .mld by the driver doesn't seem very sound. It's rather something you should specify at the cli level when you compile the source.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(The slighly annoying bit is that package cross references go through index then e.g.
{!pkg.index.mypage})

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah but no in fact we simply do:

pkg <-- empty, non materialized page created by the driver
├── Mymodule
├── index.mld
├── myimage.gif
└── page.mld

and simply decrete that index.mld is the landing page shown to users.

Note that this essentially comes back to the old --pkg option.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should focus here on what currently works (or should work!)

I think it's useful to discuss more because we'll want deep hierarchy soon and it would be a shame to create conflicts now.

furthermore it is entirely impossible to link to that page because you've no idea what the driver might have named it

Maybe it's reasonable to say that we never want to reference a parent page by its name as it was said above ?
Though, the convention should tell how it should be renamed.

In my opinion these kind of shortcuts tend to bring in more problems than they solve because now you need to say something about what happens when you have both foo.mld and foo/index.mld.

Defining what happens in this case does not seem worse than having undefined references exist.

I think good looking URLs, then good looking references come before work in the driver.

It seems assets references proposed in #1002 have some kind of search procedure (not sure it's a good idea, but why not) what if we start treating pages references like assets ?

Yes I think we want that! It's currently impossible to solve a page name conflict without limiting what can be linked to (by limiting the search path, as it's done currently).
It's also a reason why we need dot references, a page outside of this hierarchy could do {!page-pkg.index.mypage}.

We also want to do:

pkg <-- an mld page created by the driver but compiled like any other pages.
├── Mymodule
├── index.mld <-- Has a reference to 'mydir/index.mld'
├── myimage.gif
├── page.mld
└── mydir/
    ├── index.mld <-- Has a reference to 'index.mld' in the parent directory
    └── myimage.gif

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defining what happens in this case does not seem worse than having undefined references exist.

Well it goes without saying that if you use the reference it warns you for the fact it doesn't exist.

Yes I think we want that!

Actually I'm not sure. I think it would be clearer to have explicit parent notation, that is the moral equivalent of .. which we all know perfectly is sadly missing from ocaml's module system :-)

Maybe it's reasonable to say that we never want to reference a parent page by its name as it was said above ?
Though, the convention should tell how it should be renamed.

We definitively need that if we want to enable packages to escape their little island to link into other ones (in fact AFAIR that worked at some point in odoc on the --pkg option path).

pkg <-- an mld page created by the driver but compiled like any other pages.
[…]
└── mydir/
    ├── index.mld <-- Has a reference to 'index.mld' in the parent directory

But what do you put on this pkg page ? how does it materialize ?

Basically I think the idea that the parent-child relationship occurs through concrete content (i.e. a .mld page) is not necessarily a good one and we should let it go.

I think we rather want to think in terms of nested bundles: understood as a directory with .mld files, assets and possibly module files. The name of the directory defines the reference path/url prefix and the default page is index.mld so a hierarchy like:

pkg/ 
├── Mymodule
├── index.mld <-- Has a reference to 'mydir/index.mld'
├── myimage.gif
├── page.mld
└── mydir/
    ├── index.mld <-- Has a reference to 'index.mld' in the parent directory
    └── myimage.gif

Generates basically:

pkg/ 
├── Mymodule.html
├── index.html
├── myimage.gif
├── page.html
└── mydir/
    ├── index.html
    └── myimage.gif

To which you can refer to, as far as the pages are concerned and according to where you are as:

  1. In another package.
{!page-pkg.index}       # pkg/index.mld
{!page-pkg.page}        # pkg/page.mld
{!page-pkg.mydir.index} # pkg/mydir/index.mld
  1. When you are in the pkg level
{!page.index}           # pkg/index.mld
{!page.page}            # pkg/page.mld
{!page.mydir.index}.    # pkg/mydir/index.mld
  1. When ou are in the mydir level
{!page-^index}          # pkg/index.mld
{!page-^page}           # pkg/page.mld
{!page-index}           # pkg/mydir/index.mld

That way we can actually already have structured documents now. The convention for a package pkg is simply as follows:

  1. A "directory" pkg is created
  2. The content of odoc-pages is rerooted to this directory
  3. If there is no index.mld at that point, generate one for the package in pkg
  4. The modules are added to this directory.

Directory is methaphorical here this can happen through --parent options.

This is a system that is extremely easy to use for end users as it naturally maps on the file system, no crazy renamings going on behind the scene or arbitrary attachements. The position in the file system mostly defines how your reference looks like. The only thing the user needs to be made aware of is the pkg directory creation – and only if it needs to make references into other packages.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. In another package.
{!page-pkg.index}       # pkg/index.mld
{!page-pkg.page}        # pkg/page.mld
{!page-pkg.mydir.index} # pkg/mydir/index.mld

Unless you have a pkg hierarchy in your bundle in which case you need to write:

{!page-^pkg.index}       # pkg/index.mld
{!page-^pkg.page}        # pkg/page.mld
{!page-^pkg.mydir.index} # pkg/mydir/index.mld

- Other pages are children of the [index.mld] page.

If no [index.mld] is installed, it's the driver's responsibility to generate
it.

When the rendering of source code is enabled, the source tree will be named
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think any driver that follows this spec renders source today. If we're just documenting how things work right now we should omit this section.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc/driver.mld does that. This document doesn't describe what an existing driver does, rather what can be done by a driver today.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better if we document what driver should do, so that there are no useless discrepancies between them.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this paragraph should be kept then.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc/driver.mld is a bad example for this - it follows literally none of the other conventions in the paragraph, as the intent back when it was first written was to show how odoc can be used more generally than just for making docs from installed opam packages.

We haven't quite figured out a convention for the sources, and it's not at all clear to me that the hierarchy will share the same namespace as the other package pages. As a concrete example, on ocaml.org cmdliner's pages currently appear under the hierarchy https://ocaml.org/p/cmdliner/latest/doc/ - e.g. tutorial.mld is https://ocaml.org/p/cmdliner/latest/doc/tutorial.html . I can easily imagine that there would be a desire for the source to appear under https://ocaml.org/p/cmdliner/latest/src/ in which case there's no namespace clash.

I'd quite like in this PR to just focus on what's actually implemented 'in the wild' and then we can have a separate discussion on these sorts of topics to bash out the wider question about what we should do to support the new features.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree that what exists in the wild is in the scope of this PR but I removed the paragraph to not interferes with the future changes.

[source] and will be a child of [index.mld]. As a consequence, no page can be
named [source.mld].

This convention is followed by the drivers
{{:https://erratique.ch/software/odig/doc/packaging.html}Odig}
and {{:https://github.com/ocaml-doc/voodoo}ocaml.org}
and by the build system {{:https://github.com/ocaml/dune}Dune}.