[MISC] Update docs + small fixes

Aedial · Apr 27, 2023 · 6d62b50 · 6d62b50
1 parent 4307437
commit 6d62b50
Show file tree

Hide file tree

Showing 50 changed files with 932 additions and 674 deletions.
diff --git a/README.md b/README.md
@@ -1,39 +1,60 @@
-# novelai-api
-Python API for the NovelAI REST API
-
-This module is intended to be used by developers as a helper for using NovelAI's REST API.
-
-[TODO]: # (Add Quality Checking workflows and badges)
-
-| Category         | Badges                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
-|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| Pypi             | [![PyPI](https://img.shields.io/pypi/v/novelai-api)](https://pypi.org/project/novelai-api) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/novelai-api)](https://pypi.org/project/novelai-api) [![PyPI - License](https://img.shields.io/pypi/l/novelai-api)](https://pypi.org/project/novelai-api/) [![PyPI - Format](https://img.shields.io/pypi/format/novelai-api)](https://pypi.org/project/novelai-api/)                                                                                                                                                                                                                                                                                               |
-| Quality checking | [![Python package](https://github.com/Aedial/novelai-api/actions/workflows/python-package.yml/badge.svg)](https://github.com/Aedial/novelai-api/actions/workflows/python-package.yml) [![Python package](https://github.com/Aedial/novelai-api/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/Aedial/novelai-api/actions/workflows/codeql-analysis.yml) [![linting: pylint](https://img.shields.io/badge/linting-pylint-yellowgreen)](https://github.com/PyCQA/pylint) [![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) |
-| Stats            | [![GitHub top language](https://img.shields.io/github/languages/top/Aedial/novelai-api)](https://github.com/Aedial/novelai-api/search?l=python) ![Libraries.io dependency status for GitHub repo](https://img.shields.io/librariesio/github/Aedial/novelai-api) ![GitHub repo size](https://img.shields.io/github/repo-size/Aedial/novelai-api) ![GitHub issues](https://img.shields.io/github/issues-raw/Aedial/novelai-api) ![GitHub pull requests](https://img.shields.io/github/issues-pr-raw/Aedial/novelai-api)                                                                                                                                                                                                         |
-| Activity         | ![GitHub last commit](https://img.shields.io/github/last-commit/Aedial/novelai-api) ![GitHub commits since tagged version](https://img.shields.io/github/commits-since/Aedial/novelai-api/v0.11.2) ![GitHub commit activity](https://img.shields.io/github/commit-activity/m/Aedial/novelai-api)                                                                                                                                                                                                                                                                                                                                                                                                                              |
-
-
-### Prerequisites
-Before anything, ensure that nox is installed (pip install nox).
-For logging in, credentials are needed (NAI_USERNAME and NAI_PASSWORD). They should be passed via the environment variables (dotenv file supported).
-
-### Examples
-The examples are in the example folder. Each example is standalone and can be used as a test.
-Examples should be ran with `nox -s run -- python example/<name>.py`.
-
-Some tests can act as example. The full list is as follows :
-- decryption and re-encryption: tests/test_decrypt_encrypt_integrity_check.py
-- diverse generations: tests/test_generate.py
-- parallel generations: tests/test_generate_parallel.py
-
-### Usage
-The source and all the required functions are located in the novelai-api folder.
-The examples and tests showcase how this API should be used and can be regarded as the "right way" to use it. However, it doesn't mean one can't use the "low level" part, which is a thin implementation of the REST endpoints, while the "high level" part is an abstraction built on that low level.
-
-### Contributing
-You can contribute features and enhancements through PR. Any PR should pass the tests and the pre-commits before submission.
-
-The tests against the API can be ran with `nox -s test_api`. Note that having node.js installed is required for the test to run properly.
-/!\ WIP /!\ The tests against the mocked backend can be ran with `nox -s test_mock`.
-
-To install and run the pre-commit hook, run `nox -s pre-commit`. This hook should be installed before committing anything.
+# novelai-api
+Python API for the NovelAI REST API
+
+This module is intended to be used by developers as a helper for using NovelAI's REST API.
+
+[TODO]: # (Add Quality Checking workflows and badges)
+
+| Category         | Badges                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Pypi             | [![PyPI](https://img.shields.io/pypi/v/novelai-api)](https://pypi.org/project/novelai-api) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/novelai-api)](https://pypi.org/project/novelai-api) [![PyPI - License](https://img.shields.io/pypi/l/novelai-api)](https://pypi.org/project/novelai-api/) [![PyPI - Format](https://img.shields.io/pypi/format/novelai-api)](https://pypi.org/project/novelai-api/)                                                                                                                                                                                                                                                                                               |
+| Quality checking | [![Python package](https://github.com/Aedial/novelai-api/actions/workflows/python-package.yml/badge.svg)](https://github.com/Aedial/novelai-api/actions/workflows/python-package.yml) [![Python package](https://github.com/Aedial/novelai-api/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/Aedial/novelai-api/actions/workflows/codeql-analysis.yml) [![linting: pylint](https://img.shields.io/badge/linting-pylint-yellowgreen)](https://github.com/PyCQA/pylint) [![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) |
+| Stats            | [![GitHub top language](https://img.shields.io/github/languages/top/Aedial/novelai-api)](https://github.com/Aedial/novelai-api/search?l=python) ![Libraries.io dependency status for GitHub repo](https://img.shields.io/librariesio/github/Aedial/novelai-api) ![GitHub repo size](https://img.shields.io/github/repo-size/Aedial/novelai-api) ![GitHub issues](https://img.shields.io/github/issues-raw/Aedial/novelai-api) ![GitHub pull requests](https://img.shields.io/github/issues-pr-raw/Aedial/novelai-api)                                                                                                                                                                                                         |
+| Activity         | ![GitHub last commit](https://img.shields.io/github/last-commit/Aedial/novelai-api) ![GitHub commits since tagged version](https://img.shields.io/github/commits-since/Aedial/novelai-api/v0.10.5) ![GitHub commit activity](https://img.shields.io/github/commit-activity/m/Aedial/novelai-api)                                                                                                                                                                                                                                                                                                                                                                                                                              |
+
+
+# Usage
+Download via [pip](https://pypi.org/project/novelai-api):
+```
+pip install novelai-api
+```
+
+A full list of examples is available in the [example](/example) directory
+
+The API works through the NovelAIAPI object.
+It is split in 2 groups: NovelAIAPI.low_level and NovelAIAPI.high_level
+
+## low_level
+The low level interface is a strict implementation of the official API (<https://api.novelai.net/docs>).
+It only checks for input types via assert and output schema if NovelAIAPI.low_level.is_schema_validation_enabled is True
+
+## high_level
+The high level interface builds on the low level one for easier handling of complex settings.
+It handles many tasks from the frontend
+
+
+# Development
+All relevant objects are in the [novelai_api](novelai_api) directory. The [nox](https://pypi.org/project/nox/) package is required (`pip install nox`).
+
+## Contributing
+You can contribute features and enhancements through PR. Any PR should pass the tests and the pre-commits before submission.
+The pre-commit hook can be installed via
+```
+nox -s pre-commit
+```
+
+## Testing against the API
+[API](tests/api)
+
+## Testing against the mocked API
+| :warning: WIP, does not work yet :warning: |
+|--------------------------------------------|
+
+[Mock](tests/mock)
+
+## Docs
+To build the docs, run
+```
+nox -s build-docs
+```
+The docs will be locally viewable at docs/build/html/index.html
diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -1,5 +1,7 @@
 sphinx==6.1.3
-myst-parser
+# patched repo to work with relative links
+git+https://github.com/Aedial/MyST-Parser
+linkify-it-py
 sphinx-copybutton
 sphinx_last_updated_by_git
 sphinx-hoverxref
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -3,15 +3,22 @@
 # For the full list of built-in configuration values, see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 
+import datetime
+import inspect
 import os
 import sys
+from pathlib import Path
+from types import ModuleType
+from typing import List
+
+from sphinx.application import Sphinx
+from sphinx.ext.autodoc import Options
 
 # -- Project information -----------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
 project = "NovelAI API"
-# pylint: disable=W0622
-copyright = "2023, Aedial"  # noqa (built-in)
+copyright = f"{datetime.datetime.now().year}, Aedial"  # noqa (built-in), pylint: disable=W0622
 author = "Aedial"
 release = "0.11.6"
 
@@ -33,9 +40,16 @@
     "hoverxref.extension",
 ]
 
+autodoc_class_signature = "separated"
 autodoc_member_order = "bysource"
+autodoc_typehints_format = "fully-qualified"
+autodoc_preserve_defaults = True
+autodoc_inherit_docstrings = False
+
+extlinks = {"issue": ("https://github.com/Aedial/novelai-api/issues/%s", "[issue %s]")}
 
-extlinks = {"issue": ("https://github.com/sphinx-doc/sphinx/issues/%s", "[issue %s]")}
+myst_all_links_external = True
+myst_relative_links_base = "https://github.com/Aedial/novelai-api/tree/main/"
 
 suppress_warnings = ["myst.header", "git.too_shallow"]
 
@@ -62,3 +76,28 @@
 html_theme = "classic"
 # no asset yet
 # html_static_path = ['_static']
+
+
+# -- Hooks -------------------------------------------------------------------
+
+
+def format_docstring(_app: Sphinx, what: str, name: str, obj: ModuleType, _options: Options, lines: List[str]):
+    kwargs = {
+        "obj_type": what,
+        "obj_name": name,
+    }
+
+    try:
+        path = Path(inspect.getfile(obj))
+
+        kwargs.update(abspath=str(path.resolve()), filename=path.name, filestem=path.stem)
+    except TypeError:
+        pass
+
+    for i, line in enumerate(lines):
+        if "{" in line and "}" in line:
+            lines[i] = line.format(**kwargs)
+
+
+def setup(app):
+    app.connect("autodoc-process-docstring", format_docstring)
diff --git a/docs/source/example/example.boilerplate.rst b/docs/source/example/example.boilerplate.rst
@@ -0,0 +1,7 @@
+boilerplate
+===========
+
+.. automodule:: example.boilerplate
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/source/example/example.rst b/docs/source/example/example.rst
@@ -0,0 +1,46 @@
+example directory
+=================
+
+.. include:: ../../../example/README.md
+   :parser: myst_parser.sphinx_
+
+Content
+-------
+
+.. automodule:: example.download_modules
+
+.. automodule:: example.download_presets
+
+.. automodule:: example.download_shelves
+
+.. automodule:: example.download_stories_and_content
+
+.. automodule:: example.generate_controlnet_masks
+
+.. automodule:: example.generate_image
+
+.. automodule:: example.generate_image_test_samplers
+
+.. automodule:: example.generate_image_with_controlnet
+
+.. automodule:: example.generate_image_with_img2img
+
+.. automodule:: example.generate_text
+
+.. automodule:: example.generate_voice
+
+.. automodule:: example.login
+
+.. automodule:: example.login_with_proxy
+
+.. automodule:: example.suggest_tags
+
+.. automodule:: example.upscale_image
+
+Reference
+---------
+
+.. toctree::
+    :maxdepth: 2
+
+    example.boilerplate
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -14,7 +14,20 @@ TODO
 
 Reference
 =========
+
+novelai-api
+-----------
+
+.. toctree::
+   :maxdepth: 2
+
+   novelai_api/novelai_api
+
+
+example
+-------
+
 .. toctree::
    :maxdepth: 2
 
-   novelai_api
+   example/example
diff --git a/docs/source/novelai_api.python_utils.rst b/docs/source/novelai_api.python_utils.rst
diff --git a/docs/source/novelai_api.BanList.rst → ...ource/novelai_api/novelai_api.BanList.rst b/docs/source/novelai_api.BanList.rst → ...ource/novelai_api/novelai_api.BanList.rst
@@ -1,5 +1,5 @@
-novelai\_api.BanList module
-===========================
+novelai\_api.BanList
+====================
 
 .. automodule:: novelai_api.BanList
    :members:

diff --git a/docs/source/novelai_api.BiasGroup.rst → ...rce/novelai_api/novelai_api.BiasGroup.rst b/docs/source/novelai_api.BiasGroup.rst → ...rce/novelai_api/novelai_api.BiasGroup.rst
@@ -1,5 +1,5 @@
-novelai\_api.BiasGroup module
-=============================
+novelai\_api.BiasGroup
+======================
 
 .. automodule:: novelai_api.BiasGroup
    :members:

diff --git a/docs/source/novelai_api.GlobalSettings.rst → ...ovelai_api/novelai_api.GlobalSettings.rst b/docs/source/novelai_api.GlobalSettings.rst → ...ovelai_api/novelai_api.GlobalSettings.rst
@@ -1,5 +1,5 @@
-novelai\_api.GlobalSettings module
-==================================
+novelai\_api.GlobalSettings
+===========================
 
 .. automodule:: novelai_api.GlobalSettings
    :members:

diff --git a/docs/source/novelai_api.Idstore.rst → ...ource/novelai_api/novelai_api.Idstore.rst b/docs/source/novelai_api.Idstore.rst → ...ource/novelai_api/novelai_api.Idstore.rst
@@ -1,5 +1,5 @@
-novelai\_api.Idstore module
-===========================
+novelai\_api.Idstore
+====================
 
 .. automodule:: novelai_api.Idstore
    :members:

diff --git a/docs/source/novelai_api.ImagePreset.rst → ...e/novelai_api/novelai_api.ImagePreset.rst b/docs/source/novelai_api.ImagePreset.rst → ...e/novelai_api/novelai_api.ImagePreset.rst
@@ -1,5 +1,5 @@
-novelai\_api.ImagePreset module
-===============================
+novelai\_api.ImagePreset
+========================
 
 .. automodule:: novelai_api.ImagePreset
    :members:

diff --git a/docs/source/novelai_api.Keystore.rst → ...urce/novelai_api/novelai_api.Keystore.rst b/docs/source/novelai_api.Keystore.rst → ...urce/novelai_api/novelai_api.Keystore.rst
@@ -1,5 +1,5 @@
-novelai\_api.Keystore module
-============================
+novelai\_api.Keystore
+=====================
 
 .. automodule:: novelai_api.Keystore
    :members:

diff --git a/docs/source/novelai_api.NovelAIError.rst → .../novelai_api/novelai_api.NovelAIError.rst b/docs/source/novelai_api.NovelAIError.rst → .../novelai_api/novelai_api.NovelAIError.rst
@@ -1,5 +1,5 @@
-novelai\_api.NovelAIError module
-================================
+novelai\_api.NovelAIError
+=========================
 
 .. automodule:: novelai_api.NovelAIError
    :members:

diff --git a/docs/source/novelai_api.NovelAI_API.rst → ...e/novelai_api/novelai_api.NovelAI_API.rst b/docs/source/novelai_api.NovelAI_API.rst → ...e/novelai_api/novelai_api.NovelAI_API.rst
@@ -1,5 +1,5 @@
-novelai\_api.NovelAI\_API module
-================================
+novelai\_api.NovelAI\_API
+=========================
 
 .. automodule:: novelai_api.NovelAI_API
    :members:

diff --git a/docs/source/novelai_api.Preset.rst → ...source/novelai_api/novelai_api.Preset.rst b/docs/source/novelai_api.Preset.rst → ...source/novelai_api/novelai_api.Preset.rst
@@ -1,5 +1,5 @@
-novelai\_api.Preset module
-==========================
+novelai\_api.Preset
+===================
 
 .. automodule:: novelai_api.Preset
    :members:

diff --git a/docs/source/novelai_api.SchemaValidator.rst → ...velai_api/novelai_api.SchemaValidator.rst b/docs/source/novelai_api.SchemaValidator.rst → ...velai_api/novelai_api.SchemaValidator.rst
@@ -1,5 +1,5 @@
-novelai\_api.SchemaValidator module
-===================================
+novelai\_api.SchemaValidator
+============================
 
 .. automodule:: novelai_api.SchemaValidator
    :members:

diff --git a/docs/source/novelai_api.StoryHandler.rst → .../novelai_api/novelai_api.StoryHandler.rst b/docs/source/novelai_api.StoryHandler.rst → .../novelai_api/novelai_api.StoryHandler.rst
@@ -1,5 +1,5 @@
-novelai\_api.StoryHandler module
-================================
+novelai\_api.StoryHandler
+=========================
 
 .. automodule:: novelai_api.StoryHandler
    :members:

diff --git a/docs/source/novelai_api.Tokenizer.rst → ...rce/novelai_api/novelai_api.Tokenizer.rst b/docs/source/novelai_api.Tokenizer.rst → ...rce/novelai_api/novelai_api.Tokenizer.rst
@@ -1,5 +1,5 @@
-novelai\_api.Tokenizer module
-=============================
+novelai\_api.Tokenizer
+======================
 
 .. automodule:: novelai_api.Tokenizer
    :members:

diff --git a/docs/source/novelai_api.rst → docs/source/novelai_api/novelai_api.rst b/docs/source/novelai_api.rst → docs/source/novelai_api/novelai_api.rst
@@ -1,5 +1,5 @@
-novelai\_api package
-====================
+novelai-api package
+===================
 
 .. toctree::
    :maxdepth: 2
@@ -17,4 +17,3 @@ novelai\_api package
    novelai_api.StoryHandler
    novelai_api.Tokenizer
    novelai_api.utils
-   novelai_api.python_utils
diff --git a/docs/source/novelai_api.utils.rst → .../source/novelai_api/novelai_api.utils.rst b/docs/source/novelai_api.utils.rst → .../source/novelai_api/novelai_api.utils.rst
@@ -1,5 +1,5 @@
-novelai\_api.utils module
-=========================
+novelai\_api.utils
+==================
 
 .. automodule:: novelai_api.utils
    :members: