(trunk) CASSANDRA-16763 Create Cassandra documentation content for new website by michaelsembwever · Pull Request #1128 · apache/cassandra

michaelsembwever · 2021-08-04T19:05:43Z

No description provided.

michaelsembwever · 2021-12-08T13:46:11Z

we need to keep the doc/native_protocol_v*.spec files.

and if gen-nodetool-docs.py is removed, how is the nodetool pages getting generated…? it looks like they are not generated anymore, is that correct?

and why is there still all the .rst files under doc/source ?

ossarga · 2021-12-09T12:06:35Z

we need to keep the doc/native_protocol_v*.spec files.

The *.spec files were moved to doc/modules/cassandra/examples/TEXT/. See find output below. Do they need to live specifically in the doc directory?

[cassandra] trunk $ find ./doc -iname "*.spec"
./doc/modules/cassandra/examples/TEXT/native_protocol_v4.spec
./doc/modules/cassandra/examples/TEXT/native_protocol_v5.spec
./doc/modules/cassandra/examples/TEXT/native_protocol_v3.spec

and if gen-nodetool-docs.py is removed, how is the nodetool pages getting generated…? it looks like they are not generated anymore, is that correct?

The Python scripts to generate the nodetool pages and the Cassandra configuration pages live in the ./doc/scripts directory. See the find output below. The AsciiDoc generation script knows to look for gen-nodetool-docs.py script in this directory. If it is unable to find the script, it will download the script from: https://raw.githubusercontent.com/apache/cassandra/a47be7eddd5855fc7723d4080ca1a63c611efdab/doc/gen-nodetool-docs.py

[cassandra] trunk $ find ./doc/scripts -iname "*.py"
./doc/scripts/gen-nodetool-docs.py
./doc/scripts/convert_yaml_to_adoc.py

and why is there still all the .rst files under doc/source ?

Nice catch. That is an oversight on my part. The doc/source directory needs to be removed from this branch and the 4.0 branch as well.

ossarga · 2021-12-09T13:51:40Z

The doc/source directory is now removed.

michaelsembwever · 2021-12-09T21:29:05Z

The *.spec files were moved to doc/modules/cassandra/examples/TEXT/.

Are they displayed on the website anywhere?

If not, please keep them in their original location. Developers will want them where they are familiar.
(It would be nice to have them out on the website, but that can come later.)

The Python scripts to generate the nodetool pages and the Cassandra configuration pages live in the ./doc/scripts directory.

If ./doc/scripts/gen-nodetool-docs.py still exists, and those pages are generated, we should keep the generated pages out of git.

I'm presuming that all the .adoc pages under doc/modules/cassandra/pages/tools/nodetool/ are generated?

ossarga · 2021-12-10T01:53:28Z

Are they displayed on the website anywhere?

@polandll I am unsure about the answer to this. Do you know by any chance?

If not, please keep them in their original location. Developers will want them where they are familiar.
(It would be nice to have them out on the website, but that can come later.)

@polandll If we move the *.spec files back to their original location will anything break?

If ./doc/scripts/gen-nodetool-docs.py still exists, and those pages are generated, we should keep the generated pages out of git.

Yes, we can do this.

When we do need to make sure that folks are aware that they will need to do one of two things when generating the website HTML;

call ./run.sh docs ... to generate the ADOC files prior to calling ./run.sh build ... to render the website HTML, or
call ./run.sh build -g ... to generate the ADOC as part of the process to generate the website HTML.

I'm presuming that all the .adoc pages under doc/modules/cassandra/pages/tools/nodetool/ are generated?

Yes, they are generated using gen-nodetool-docs.py script.

polandll · 2021-12-10T20:34:14Z

You can move the *.spec back to their original location @ossarga. They were just part of the non-version-specific files that I moved to cassandra-website. I don't think they are referenced on any webpage, not I'm not 100% sure.

michaelsembwever · 2021-12-11T18:02:30Z

They were just part of the non-version-specific files that I moved to cassandra-website. I don't think they are referenced on any webpage, not I'm not 100% sure.

They are used by developers, and drivers. It would nice to have them visible on the website, but they are exact documents the developers rely upon as-is.
We should keep them in-tree for now, because developers need them versioned against the code (though it might be worthy of a dev@ ML discussion).

This commit sets up a new documentation structure and format: * The directory struture changes from a Sphinx project to an Antora module layout. * The formatting of the content changes from reStructuredText to AsciiDoc. The documentation must now be built and published with Antora. This change was done to make maintaining versioning documentation easier. As Antora is designed to generate versioned documentation. The old directory structure was: <ROOT> - doc/ - cql3/ - CQL.css - CQL.textile - source/ - _static/ - _templates/ - _theme/ - _util/ - <other directory sections>/ - conf.py - index.rst - <other *.rst pages> - make.bat - Makefile - README.md - SASI.md - <*.spec files> - <generation scripts> The new directory structure organises the documentation into modules: <ROOT> - doc/ - cql3 - modules/ - cassandra/ - assets/ - examples/ - pages/ - partials/ - nav.adoc - ROOT/ - pages/ - nav.adoc - scripts/ - antora.yaml - README.md - SASI.md - <*.spec files> Patch by Lorina Poland; reviewed by Michael Semb Wever and Anthony Grasso for CASSANDRA-16763

* Fixed AsciiDoc link errors * Deleted Docker files which ran Sphinx * Deleted Make files * Deleted Python scripts which generated RST files as they have been replaced with scripts in the 'scripts' directory that generate ADOC files * Removed configuration documentation file - doc/modules/cassandra/pages/configuration/cass_yaml_file.adoc * Removed all nodetool command (.txt) and documentation (*.adoc) files - doc/modules/cassandra/examples/TEXT/NODETOOL/*.txt - doc/modules/cassandra/pages/tools/nodetool/*.adoc - doc/modules/cassandra/pages/tools/nodetool/nodetool.txt * Removed legacy .rst -> .adoc conversion script - doc/modules/cassandra/pages/tools/nodetool/convert.sh Patch by Anthony Grasso; reviewed by Michael Semb Wever for CASSANDRA-16763 Co-authored-by: polandll <lorina@datastax.com>

* Added documentation files to RAT exclude list: - doc/antora.yaml - doc/modules/* * Added generated doc files to .gitignore: - doc/modules/cassandra/pages/configuration/cass_yaml_file.adoc - doc/modules/cassandra/pages/tools/nodetool/ - doc/modules/cassandra/examples/TEXT/NODETOOL/ Patch by Anthony Grasso; reviewed by Michael Semb Wever for CASSANDRA-16763

michaelsembwever · 2021-12-23T20:02:09Z

doc/README.md

 The documentation can then be built from this directory by calling `make html`
-(or `make.bat html` on windows). Alternatively, the top-level `ant gen-doc`
-target can be used.  When using Python 3.6 on Windows, use `sphinx_build -b html source build`.
+Alternatively, the top-level `ant gen-doc` target can be used.  


ant gen-doc no longer works. (because doc/Makefile is removed)

There is now way to generate docs from the cassandra directory (everything is now done here).

Is there a minimum doc/Makefile we can put in place so that ant gen-doc runs in a basic form, for the sake of catching breakages…?

In fact… can we move some of that docker-entrypoint.sh back into doc/Makefile ?

see
polandll#16
and
apache/cassandra-website#86

Good catch. I guess we have a Makefile that does part of what the generate_cassandra_versioned_docs function in the docker-entrypoint.sh does. That is, build the jar files, then call the nodetool and cassandra configuration adoc generation scripts. This would be the most that gen-doc could do and should be enough to weed out any breakages. The README also needs to be updated. We would need to make this change on all three branches.

As a bonus, we could probably update the generate_cassandra_versioned_docs function to call ant gen-doc.

Yup, covered in the PRs above.

But… i'm going to change the Makefile and ant targets for compatibility reasons.

Generating in-tree docs: ant gen-doc and make html are for a full antora build of the in-tree docs. This won't be implemented in these PRs, but will be reserved for when it is. (We had previously spoken about being able to create non-styled antora docs, even something that can be put into man pages.)

New targets for generating the dynamic adoc pages: ant gen-asciidoc and make gen-asciidoc.
The former will be called from the generate_cassandra_versioned_docs function.

PRs updated accordingly. (#16 needs to be applied to three branches.)

There's another commit for trunk in polandll#18 (i botched the squash me commit you merged in already)

PR for cassandra-4.0 polandll#17

PR for cassandra-3.11 polandll#19

…ate README.md too.

michaelsembwever · 2022-01-05T23:25:00Z

committed with 54aee87

michaelsembwever changed the title ~~CASSANDRA-16763 Create Cassandra documentation content for new website~~ (trunk) CASSANDRA-16763 Create Cassandra documentation content for new website Aug 4, 2021

ossarga force-pushed the trunk branch 2 times, most recently from 1345977 to 39782a5 Compare October 8, 2021 04:39

ossarga force-pushed the trunk branch from 39782a5 to 21fd477 Compare October 15, 2021 23:46

ossarga force-pushed the trunk branch 6 times, most recently from 420dba9 to 87f4e57 Compare December 7, 2021 13:02

ossarga force-pushed the trunk branch from 87f4e57 to 768dcc7 Compare December 9, 2021 13:41

ossarga force-pushed the trunk branch from 768dcc7 to 4fbd318 Compare December 19, 2021 11:12

polandll and others added 3 commits December 20, 2021 23:02

ossarga force-pushed the trunk branch from 4fbd318 to b3840e8 Compare December 20, 2021 13:17

michaelsembwever commented Dec 23, 2021

View reviewed changes

SQUASH ME - make ant gen-doc work again (in a preliminary way). upd…

0d6f4e8

…ate README.md too.

michaelsembwever closed this Jan 5, 2022

Conversation

michaelsembwever commented Aug 4, 2021

Uh oh!

michaelsembwever commented Dec 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ossarga commented Dec 9, 2021

Uh oh!

ossarga commented Dec 9, 2021

Uh oh!

michaelsembwever commented Dec 9, 2021

Uh oh!

ossarga commented Dec 10, 2021

Uh oh!

polandll commented Dec 10, 2021

Uh oh!

michaelsembwever commented Dec 11, 2021

Uh oh!

michaelsembwever Dec 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelsembwever Dec 23, 2021

Choose a reason for hiding this comment

Uh oh!

ossarga Dec 23, 2021

Choose a reason for hiding this comment

Uh oh!

michaelsembwever Dec 24, 2021

Choose a reason for hiding this comment

Uh oh!

michaelsembwever Dec 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelsembwever Dec 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelsembwever Dec 30, 2021

Choose a reason for hiding this comment

Uh oh!

michaelsembwever commented Jan 5, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

michaelsembwever commented Dec 8, 2021 •

edited

Loading

michaelsembwever Dec 23, 2021 •

edited

Loading

michaelsembwever Dec 24, 2021 •

edited

Loading

michaelsembwever Dec 30, 2021 •

edited

Loading