Load the glossary from the Octez docs

[timothymcmackin]

Grab the glossary from the Octez docs, use the script they provide to extract the content, and overwrite the existing glossary.md file.

I tried editing the output HTML file after the build, but in that case Docusaurus was caching the old version of the file. Based on discussion with some colleagues, it appears to be better to do this manipulation before the build.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
docs-staging	✅ Ready (Inspect)	Visit Preview	Jul 25, 2024 8:40pm

[NicNomadic]

Looks great! I generated the glossary and built the doc locally, and everything seems fine.
The only little thing I noticed is that external links such as https://tezos.gitlab.io/paris/consensus.html#slashing-paris are not open in a separate tab, nor do they show up with the little weblink icon. But maybe that's not important.

[timothymcmackin]

Looks great! I generated the glossary and built the doc locally, and everything seems fine. The only little thing I noticed is that external links such as https://tezos.gitlab.io/paris/consensus.html#slashing-paris are not open in a separate tab, nor do they show up with the little weblink icon. But maybe that's not important.

Made external links open in a new tab and show the external link icon.

[NicNomadic]

Made external links open in a new tab and show the external link icon.

Strangely, I see now an empty page with just a link to the Octez glossary, which indeed opens in a new tab. So this could also work, but I think it's not what you're trying to do, right?

[timothymcmackin]

Made external links open in a new tab and show the external link icon.

Strangely, I see now an empty page with just a link to the Octez glossary, which indeed opens in a new tab. So this could also work, but I think it's not what you're trying to do, right?

There's a glitch in docusaurus; when I edit the page after the build, the docusaurus serve command shows the old page -- it caches it somehow. When we deploy to AWS, only the built output gets pushed, not the cache, so I think this will work. I'm investigating the cache issue.

[timothymcmackin]

@NicNomadic

Strangely, I see now an empty page with just a link to the Octez glossary,

I've reworked this PR to edit the glossary file before the build to avoid the docusaurus caching issue. Now you can see the imported glossary in the preview: https://docs-staging-git-include-html-trili-tech.vercel.app/overview/glossary

[NicNomadic]

Beware, you import the glossary for Alpha, but you should import the active protocol's instead (from https://tezos.gitlab.io/active/glossary.html).

[NicNomadic]

The terms no longer appear in bold, it was nicer before!

[NicNomadic]

Clicking on a term goes a bit after its definition (both in Safari and Firefox), is this hard to fix?

[NicNomadic]

Links to the pages within the Octez, such as "Slashing" or "application time" don't work. Their URL contains "...//..." instead of ".../active/...".

[timothymcmackin]

The terms no longer appear in bold,

Fixed, thx for catching that regression.

import the active protocol's instead

Fixed.

Their URL contains "...//..." instead of ".../active/...".

I see this in the output of the extract_content script -- could be a bug on that side?

Clicking on a term goes a bit after its definition

Investigating...

[NicNomadic]

Their URL contains "...//..." instead of ".../active/...".
I see this in the output of the extract_content script -- could be a bug on that side?

This issue should be fixed when merging https://gitlab.com/tezos/tezos/-/merge_requests/13987. Or use already the script in that branch: https://gitlab.com/nomadic-labs/tezos/-/raw/nic@global-glossary-doc/docs/scripts/extract_content.

[timothymcmackin]

Clicking on a term goes a bit after its definition

I don't know why this is happening so I added a custom script to add some space after the user clicks an anchor on the page. It won't work if the user comes to the page directly via a URL.

[timothymcmackin]

This issue should be fixed...

Yes, that new script seems to work. It leaves only a few URLs like this, which look odd but work:
https://tezos.gitlab.io/active/../shell/validation.html#shell-header

[NicNomadic]

Yes, that new script seems to work.

I wanted to validate your last version, but when I run npm run build it stops with an error:

Used Octez docs script to pull the content from its glossary
/Users/nic/github/tezos-developer-docs/src/scripts/process_downloaded_glossary.js:86
  const h1 = trimmed.querySelector('h1');
                     ^
TypeError: Cannot read properties of null (reading 'querySelector')
    at process_glossary (/Users/nic/github/tezos-developer-docs/src/scripts/process_downloaded_glossary.js:86:22)

Any suggestion?

[timothymcmackin]

Any suggestion?

@NicNomadic An element changed in the glossary. Load my updated version and it should work now.

[NicNomadic]

I checked that most of the issues I raised have been solved, including:

• The terms no longer appear in bold,
• import the active protocol's instead [of Alpha]
• Clicking on a term goes a bit after its definition

However, it remains the following one:

Links to the pages within the Octez, such as "Slashing" or "application time" don't work. Their URL contains "...//..." instead of ".../active/...".

Now links no more contain "...//...", but they don't have the right prefix, which should be https://tezos.gitlab.io/active/ instead of http://localhost:3000/overview/. As a result, clicking on those links results in a 404 error page.

[timothymcmackin]

As a result, clicking on those links results in a 404 error page.

This build process runs the conversion script at https://gitlab.com/tezos/tezos/-/raw/master/docs/scripts/extract_content?ref_type=heads&inline=false, which should fix the links. I'll investigate.

[timothymcmackin]

@NicNomadic This seems to work: I've taken the two find-replace commands from your extract_content script and handled them myself so I don't have to bother calling awk. Now the external and internal links seem to work. This is starting to seem like a fragile solution, though. I wonder if it would be better if I got the raw RST and ran it through pandoc or something to get markdown.

https://docs-staging-iibre089k-trili-tech.vercel.app/overview/glossary

[NicNomadic]

@NicNomadic This seems to work [...]. This is starting to seem like a fragile solution, though. I wonder if it would be better if I got the raw RST and ran it through pandoc or something to get markdown.

I agree that this is a low-cost and somewhat fragile solution, but I tried with Pandoc and it didn't handle the advanced RST combinations used in the glossary, so for the least this other solution requires more work to set up. What if we start with the current simple solution as long as it works (modulo simple adaptations), and if this becomes too difficult later on, we can explore further the Pandoc-based track?

[NicNomadic]

LGTM