Feat rspy138/architecture documentation

.github/actions/build-technical-doc/action.yml

@@ -17,6 +17,6 @@ runs:
 
     - name: Generate technical documentation
       run: |
-        poetry run asciidoxy --base-dir doc --build-dir ../dist/doc/ --multipage doc/index.adoc
+        poetry run asciidoxy --warnings-are-errors --base-dir doc --image-dir images --build-dir ../dist/doc/ --multipage doc/index.adoc
       working-directory: ./docs
       shell: bash

docs/doc/dev/background/workflow.adoc

@@ -2,7 +2,7 @@
 
 == Git
 
-The project is using the https://git-flow.readthedocs.io/fr/latest/presentation.html[gitflow].
+The project is using the https://git-flow.readthedocs.io/en/latest/presentation.html[gitflow].
 
 The feature branches are named following the pattern "feat-<jira-id>/<short-description>"
 For example : "feat-rspy31/init-tech-doc"

docs/doc/dev/design/services/design.adoc

@@ -33,8 +33,8 @@ It is handled as a python project as if it is a rs-server service.
 
 == Services
 
-:leveloffset: +3
+:leveloffset: +2
 
 include::cadip.adoc[]
 
-:leveloffset: -3
+:leveloffset: -2

docs/doc/dev/doc-generation/how-to.adoc

@@ -1,28 +1,29 @@
 = Generate rs-server technical documentation
+:sectnums:
 
 This procedure is run automatically in the CI by the 'generate documentation' workflow.
 It can be executed locally to verify the generated documentation (before publishing it for example).
 
-See ${include("description.adoc")} for more details on the process and technical stuff.
+See ${cross_document_ref("description.adoc")} for more details on the process and technical stuff.
 
-. Pre-requises
+== Prerequisites
 
 The local developer environment has been setup.
 The global python project is setup.
 The python sub-project for doc generation is also setup.
 
-. Generate the technical documentation
+== Generate the technical documentation
 
 From the rs-server/docs folder, execute the command:
 [source, bash, indent=0]
 ----
-poetry run asciidoxy --base-dir doc --build-dir ../dist/doc/ --multipage doc/index.adoc
+poetry run asciidoxy --warnings-are-errors --base-dir doc --image-dir images --build-dir ../dist/doc/ --multipage doc/index.adoc
 ----
 
 This command generates the html pages of all the technical documentation
 and write them in the rs-server/dist/doc folder.
 
-. Generate the python api documentation
+== Generate the python api documentation
 
 From the root folder, run the following command :
 [source, bash, indent=0]
@@ -43,7 +44,7 @@ poetry run sphinx-build -M html docs/doc/api/python/ dist/doc/output/api/python/
 It generates the html pages for the python api of the rs-server.
 The generated api is accessible locally at 'dist/doc/api/html/index.html'.
 
-. Generate the REST API documentation
+== Generate the REST API documentation
 
 From the root folder, run the following command :
 [source, bash, indent=0]
@@ -69,7 +70,7 @@ The output folder is the one expected for integration with the main document of
 NOTE: This is a POC waiting for the frontend to be implemented.
 The documented REST API will be the one of the frontend at the end.
 
-. Verify the generated documentation
+== Verify the generated documentation
 
 The technical documentation is generated in the folder "dist/doc/output/".
 The main page is the file "index.html" is this folder.

docs/doc/dev/environment/description.adoc

@@ -46,7 +46,7 @@ and explains briefly the choices that have been made.
 |commonly used by the team
 
 |security check
-|trivi
+|trivy
 |used in the previous phase
 
 |technical documentation

docs/doc/dev/environment/installation.adoc

@@ -1,10 +1,10 @@
 = Install your environment
 
-== Pre requises
+== Prerequisites
 
 This tutorial assumes the developer has already basic knowledge
 and is using a ubuntu working station.
-This tutorial lets the user uses its preferred IDE.
+This tutorial lets the user uses their preferred IDE.
 You could extend your environment with specific IDE integration.
 
 == Checkout the project
@@ -72,7 +72,7 @@ You should first initialize the project locally
 cd $RSPY_ROOT/src/rs-server
 poetry install --with dev
 ----
-This command initialize and activate a virtual environment for your project.
+This command initializes and activates a virtual environment for your project.
 It installs in this environment the project dependencies
 and the develop dependencies (--with dev).
 
@@ -93,15 +93,15 @@ pytest tests -m "unit"
 
 NOTE: Your IDE can provide integration to run your unittests.
 
-== Install trivi
+== Install trivy
 
-link:https://aquasecurity.github.io/trivy/v0.47/[trivy] is used to do some security and license checks on the repository, the generated wheel packages and the docker images.
+link:https://aquasecurity.github.io/trivy/latest/[trivy] is used to perform security and license checks on the repository, the generated wheel packages and the docker images.
 
-trivi is run on the pre-commit and the CI
+trivy is run on the pre-commit and the CI
 to verify the repository compliance with common vulnerabilities, secrets and licenses.
 
 To install trivy on your local environment,
-follow link:https://aquasecurity.github.io/trivy/v0.47/getting-started/installation/[the official procedure].
+follow link:https://aquasecurity.github.io/trivy/latest/getting-started/installation/[the official procedure].
 
 == Install pre-commit hooks
 
@@ -115,7 +115,8 @@ poetry run pre-commit install
 ----
 
 Once installed, checks will be performed before each commit
-to ensure your code is compliant with project best practice :
+to ensure your code is compliant with project best practices :
+
 * assert no unresolved merge conflicts
 * fix end of file
 * fix trailing whitespace
@@ -137,6 +138,6 @@ to keep your code compliant with the coding style at any moment.
 
 == The next steps
 
-The ${include("description.adoc")} can provide you with more details about the environment.
-The ${include("../background/workflow.adoc")} describes the workflow followed by developers to implement stories.
-You may also want to read the ${include("../code-style.adoc")}.
+The ${cross_document_ref("description.adoc")} can provide you with more details about the environment.
+The ${cross_document_ref("../background/workflow.adoc")} describes the workflow followed by developers to implement stories.
+You may also want to read the ${cross_document_ref("../code-style.adoc")}.

docs/doc/index.adoc

@@ -2,23 +2,26 @@
 
 == User manual
 
-The main functionalities are explained here :
+The overall architecture is described here:
+${include("users/architecture.adoc")}
+
+The main functionalities are explained here:
 ${include("users/functionalities.adoc")}
 
 
 == Developer manual
 
 All you want to know when you are a rs-server developer.
 
-The documentation is organized in 4 sections :
+The documentation is organized in 4 sections:
 
-* Tutorials to help you start coding on rs-server :
+* Tutorials to help you start coding on rs-server:
 ** ${include("dev/environment/installation.adoc")}
 
 * How to guides to provide practical common procedures
 ** ${include("dev/doc-generation/how-to.adoc")}
 
-* Reference guides give you technical information about the code and software API :
+* Reference guides give you technical information about the code and software API:
 ** ${include("dev/code-style.adoc")}
 ** ${include("api/python/html/index.adoc")}
 ** ${include("api/rest/index.adoc")}

docs/doc/users/architecture.adoc

@@ -0,0 +1,37 @@
+= Architecture
+
+== Static View
+
+We can split the system into following Components:
+
+* *RS-Server*: The RS-Server controls user access to all sensitive interfaces: Catalog, LTA, ADGS, PRIP, CADIP. As a consequence, we can distinguish following components:
+
+** *RS-Server Frontend*
+** *RS-Server Backend / Catalog*
+** *RS-Server Backend / PRIP*
+** *RS-Server Backend / CADIP*
+** *RS-Server Backend / LTA*
+** *RS-Server Backend / AUXIP*
+
+* *RS-Server libraries*: this is a set of python functions that provide processing task and flows. Flows and tasks can be orchestrated by Prefect server or executed from any Python environment. On RS-Server side there will be also function to compute performance indicator.
+
+* *RS-Virtual environment*: Component that gives users access to computational environments and resources for executing processing chains.
+
+* *Processing*: this group hosts all components to process Sentinel products. The CFI are not part of the group. Only wrappers, preparation worker and Dask cluster to provide processing power. Here are the components:
+
+** *Dask cluster*
+** *DPR libraries*
+
+* *Infrastructure*: The infrastructure is a portable, extensible and open-source platform orchestrated by Kubernetes. We can isolate two specific layout which are monitoring and security. As a consequence, we can distinguish following components:
+
+** *Infrastructure core*
+** *Infrastructure monitoring*
+** *Infrastructure security*
+
+image::staticview.png[]
+
+== Dynamic View
+
+The following schema highlights main interactions between the components.
+
+image::dynamicview.png[]

docs/doc/users/functionalities.adoc

@@ -7,7 +7,7 @@ The CADU Interface delivery Point (CADIP) is a pick-up point for Sentinel CADU d
 clients to straightforwardly discover and retrieve available data files through a standard OData RESTful API. The following endpoints have been implemented in RS-Server to interact with CADIP RESTful API
 [[cadip-search]]
 
-== [.green]#Search Endpoint#
+=== [.green]#Search Endpoint#
 This endpoint retrieves a list of products from the CADU system for a specified station within a given time range and return a STAC compatible FeatureCollection response.
 
 ==== API Reference
@@ -53,7 +53,7 @@ GET /cadip/station123/cadu/search?datetime=2023-01-01T00:00:00Z/2023-01-02T23:59
 }
 
 [[cadip-download]]
-== [.green]#Download Endpoint#
+=== [.green]#Download Endpoint#
 
 This endpoint initiates an asynchronous download process for a CADU product using EODAG. If specific parameters are provided, endpoint also upload the file to an S3 bucket.
 
@@ -93,7 +93,7 @@ GET /cadip/station123/cadu?name=DCS_04_S1A_20231121072204051312_ch1_DSDB_00001.r
 ----
 
 [[cadip-status]]
-== [.green]#Status Endpoint#
+=== [.green]#Status Endpoint#
 This endpoint is used to query the download status of an CADU file.
 
 ==== API Reference
@@ -127,7 +127,7 @@ GET /cadip/{station}/cadu/status?name=DCS_04_S1A_20231121072204051312_ch1_DSDB_0
 The Auxiliary Data Gathering Service (ADGS) is a pick-up point for Sentinel auxiliary files. This service allows
 clients to discover and retrieve available auxiliary data files through a standard OData RESTful API. The following endpoints have been implemented in RS-Server to interact with ADGS RESTful API.
 [[adgs-search]]
-== [.green]#Search Endpoint#
+=== [.green]#Search Endpoint#
 
 This endpoint handles the search for products in the AUX station within a specified time interval and return a STAC compatible FeatureCollection response.
 
@@ -169,7 +169,7 @@ GET /adgs/aux/search?datetime=2018-01-01T00:00:00Z/2023-01-02T23:59:59Z&limit=10
 }
 
 [[adgs-download]]
-== [.green]#Download Endpoint#
+=== [.green]#Download Endpoint#
 
 This endpoint initiates an asynchronous download process for an AUX product using EODAG. If specific parameters are provided, endpoint also upload the file to an S3 bucket.
 
@@ -208,7 +208,7 @@ GET /adgs/aux?name=S2__OPER_AUX_ECMWFD_PDMC_20190216T120000_V20190217T090000_201
 ----
 
 [[adgs-status]]
-== [.green]#Status Endpoint#
+=== [.green]#Status Endpoint#
 This endpoint is used to query the download status of an AUX file.
 
 ==== Endpoint

docs/doc/users/installation.adoc

@@ -2,7 +2,7 @@
 
 NOTE: The rs-server installation will be described here
 
-// TODO pre requises to install rs-server
+// TODO prerequisites to install rs-server
 // TODO the install procedure
 // TODO what is expected as a result
 // TODO how to check it is nominally installed