gtk-doc output varies from parallelism

[bmwiedemann]

While working on reproducible builds for openSUSE, I found that
our buzztrax package varies unless we build it with make -j1
without parallelism.

 <a name="BtCmdApplication.object-hierarchy"></a><h2>Object Hierarchy</h2>
 <pre class="screen">    <a href="/usr/share/gtk-doc/html/gobject/gobject-The-Ba
se-Object-Type.html#href_anchor">GObject</a>
-    <span class="lineart">╰──</span> BtApplication
+    <span class="lineart">╰──</span> <a href="../html/BtApplication.html#href_a
nchor">BtApplication</a>
         <span class="lineart">╰──</span> BtCmdApplication
 </pre>

These files are created by gtkdoc-fixxref called by make -C docs/reference/bt-cmd

It his something that can be fixed in buzztrax, or do we need to work with gtk-doc maintainers?

[ensonic]

Can't be fixed in gtk-doc (funny I am the gtk-doc maintainer). The docs could be build in paralell, but the xrefs have to be built in order. For now just don't built them in parallel if it doesn't work :).

[bmwiedemann]

Would be good, to have better documentation on how to do such a non-parallel build.

Currently we build the whole buzztrax with make -j4 (number depending on number of cores in build machine) and to make it reproducible, we could build it with make, but then all C code is also built without parallelism - increasing the total build time from 267 to 542 seconds.

One ugly hack could be to build everything with parallelism, then remove all docs and rebuild without parallelism. Not nice.

A better way would be to run non-parallel make docs as first step, but then the build fails with

In file included from src/lib/core/core.c:27:
src/lib/core/core_private.h:60:10: fatal error: marshal.h: No such file
 or directory

[bmwiedemann]

Hi, I'd still welcome help on how to get make docs to work before building everything (in parallel)

[bmwiedemann]

ping

[ensonic]

Sorry for the delay. My computer is so fast and the code base is so small, that I built with make (no -j4). I think we could try to setup some dependencies between the doc modules here:
https://github.com/Buzztrax/buzztrax/blob/master/Makefile.am#L90

I'll see if I can give this a try later this week.

[ensonic]

Again sorry for the delay. Please reopen if the change turns out to be insufficient.

[bmwiedemann]

I tried with master version, but still get diffs between -j1 and -j4 from gtkdoc-fixxref ?

+++ new//usr/share/gtk-doc/html/buzztrax-edit/BtPianoKeys.html  2021-02-28 04:23
:36.570132024 +0000
@@ -136,7 +136,7 @@
 <a name="BtPianoKeys-key-pressed"></a><h3>The <code class="literal">“key-presse
d”</code> signal</h3>
 <pre class="programlisting"><span class="returnvalue">void</span>
 user_function (<a class="link" href="BtPianoKeys.html" title="BtPianoKeys"><span class="type">BtPianoKeys</span></a> *self,
-               <a href="../html/buzztrax-gst-GstBtMusicEnums.html#href_anchor"><span class="type">GstBtNote</span></a>    key,
+               <span class="type">GstBtNote</span>    key,

and make docs fails with

 libtool: compile:  gcc ... -c src/lib/core/core.c -fPIC -DPIC -o src/lib/core/.libs/libbuzztrax_core_la-core.o
 In file included from src/lib/core/core.c:27:
 src/lib/core/core_private.h:60:10: fatal error: marshal.h: No such file or directory
    60 | #include "marshal.h"
       |          ^~~~~~~~~~~
 compilation terminated.

[ensonic]

Weird.This is what I did to test things:

# terminal 1
git clone buzztrax buzztrax.doc_determinism
# terminal 2 
cd buzztrax
make clean; make -j1
# terminal 3
cd buzztrax.doc_determinism
make clean; make -j4
# terminal 1
diff buzztrax{,.doc_determinism}/docs/reference/bt-edit/html/BtPianoKeys.html

and they are the same. Installing the docs should not change anything. Install only rebases the links. The fixxrefs step is parts of html-build. I also did a make install and see no difference:

diff ${HOME}/buzztrax{,_doc_determinism}/share/gtk-doc/html/buzztrax-edit/BtPianoKeys.html

Regarding the make docs issue, was that after make clean? And was that with '-j4' ? Tried all combinations and it did not happen for me :/

[ensonic]

FYI: maybe we can find some time interactively figure this in https://gitter.im/Buzztrax/buzztrax