From 12abfc4cdbad493838d96428eb255b6b7b5d20b3 Mon Sep 17 00:00:00 2001 From: Francis Charette Migneault Date: Wed, 23 Sep 2020 18:02:54 -0400 Subject: [PATCH 1/3] add docs example references (relates to #61) --- CHANGES.rst | 1 + docs/source/examples.rst | 33 +++++++++++++++++++++++++++++++++ docs/source/faq.rst | 6 ++++++ docs/source/package.rst | 2 +- docs/source/processes.rst | 5 +++++ docs/source/references.rst | 13 +++++++++++++ docs/source/toc.rst | 1 + weaver/__meta__.py | 2 -- 8 files changed, 60 insertions(+), 3 deletions(-) create mode 100644 docs/source/examples.rst diff --git a/CHANGES.rst b/CHANGES.rst index 508f504df..8bcab87de 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -9,6 +9,7 @@ Changes Changes: -------- - Add ``data`` input support for `CWL` `Workflow` step referring to `WPS-3 Process`. +- Add documentation example references to `Application Package` and `Process` ``Deploy``/``Execute`` repositories. Fixes: ------ diff --git a/docs/source/examples.rst b/docs/source/examples.rst new file mode 100644 index 000000000..11398f824 --- /dev/null +++ b/docs/source/examples.rst @@ -0,0 +1,33 @@ +.. _examples: +.. include:: references.rst + +******************************************** +Process and Application Package Examples +******************************************** + +The principal source of examples can be found directly within |weaver-func-test-apps|_. Most :term:`Process` listed +in this repository employ an :term:`Application Package` defined using a locally provided :term:`CWL`. Some other +cases will refer to pre-deployed :term:`WPS` processes. Refer to corresponding |deploy-req-name|_ request payloads, +also provided in that location, that will indicate the kind of :term:`Process` definition employed through their +``executionUnit`` specification. Once successfully deployed on a :term:`ADES` or :term:`EMS`, the also provided +|exec-req-name|_ request body can be employed as format reference to run the operation (input values can be modified). + +The general naming convention is: + + - ``DeployProcess_.json`` for the |deploy-req-name|_ request payload. + - ``Execute_.json`` for the |exec-req-name|_ request payload. + - ``.cwl`` for the :term:`CWL` :term:`Application Package` when applicable. + +.. note:: + There can be minor variations (camel/snake case, upper/lower case) of the exact ```` employed in the + file names according to the different reference repositories. + + +Further examples are also available following `OGC Testbeds` developments. The produced applications in this case can +be found in |ogc-testbeds-apps|_. In that repository, the above naming convention will sometime be employed, but each +:term:`Process` is contained within its own sub-directory. Another naming conversion is sometime used, although the +files provide equivalent details: + + - ``/deploy.json`` for the |deploy-req-name|_ request payload. + - ``/execute.json`` for the |exec-req-name|_ request payload. + - ``/package.cwl`` for the :term:`CWL` :term:`Application Package` when applicable. diff --git a/docs/source/faq.rst b/docs/source/faq.rst index bcee2ae9a..889da6f56 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -121,4 +121,10 @@ continuously improved by the developers (including being updated according to ch - |cwl-guide|_ - |cwl-cmdtool|_ - |cwl-workflow|_ +- See also below. + +Where can I find examples of Process Deployment, Execution and Application Package definitions? +===================================================================================================== + +Many examples are reported in detail in the |examples|_ chapter. diff --git a/docs/source/package.rst b/docs/source/package.rst index 34b48661a..bd44b8da8 100644 --- a/docs/source/package.rst +++ b/docs/source/package.rst @@ -132,7 +132,7 @@ preserve and enforce matching input/output ``format`` definition in both :term:` not introduce ambiguous results (see :ref:`File Format` for more details). -Correspondance between CWL and WPS fields +Correspondence between CWL and WPS fields =========================================== Because :term:`CWL` definition and :term:`WPS` process description inherently provide "duplicate" information, many diff --git a/docs/source/processes.rst b/docs/source/processes.rst index 46e782ff3..5b569c01c 100644 --- a/docs/source/processes.rst +++ b/docs/source/processes.rst @@ -19,6 +19,11 @@ Each one of them are accessible through the same API interface, but they have di - `Remote Provider`_ +.. seealso:: + Section |examples|_ provides multiple concrete use cases of `Deploy`_ and `Execute`_ request payloads + for diverse set of applications. + + Builtin ------- diff --git a/docs/source/references.rst b/docs/source/references.rst index df9d425fd..b59fb7b48 100644 --- a/docs/source/references.rst +++ b/docs/source/references.rst @@ -61,6 +61,14 @@ .. |submit-issue| replace:: submit a new issue .. _submit-issue: https://github.com/crim-ca/weaver/issues/new/choose +.. Example references +.. |examples| replace:: Examples +.. _examples: examples.rst +.. |weaver-func-test-apps| replace:: Weaver functional tests +.. _weaver-func-test-apps: https://github.com/crim-ca/weaver/tree/master/tests/functional/application-packages +.. |ogc-testbeds-apps| replace:: OGC-Testbeds Applications +.. _ogc-testbeds-apps: https://github.com/crim-ca/application-packages + .. External references .. _Celery: https://docs.celeryproject.org/en/latest/ .. _Gunicorn: https://gunicorn.org/ @@ -79,12 +87,17 @@ .. _email-template: ../../../weaver/wps_restapi/templates/notification_email_example.mako .. API requests +.. Full path displayed, otherwise use '-name' suffixed reference for same link with only the general name (no path) +.. |deploy-req-name| replace:: Deploy +.. _deploy-req-name: `deploy-req`_ .. |deploy-req| replace:: ``POST {WEAVER_URL}/processes`` (Deploy) .. _deploy-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/Processes%2Fpaths%2F~1processes%2Fpost .. |getcap-req| replace:: ``GET {WEAVER_URL}/processes`` (GetCapabilities) .. _getcap-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/Processes%2Fpaths%2F~1processes%2Fget .. |describe-req| replace:: ``GET {WEAVER_URL}/processes/{id}`` (DescribeProcess) .. _describe-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/Processes%2Fpaths%2F~1processes~1%7Bprocess_id%7D~1package%2Fget +.. |exec-req-name| replace:: Execute +.. _exec-req-name: `exec-req`_ .. |exec-req| replace:: ``POST {WEAVER_URL}/processes/{id}/jobs`` (Execute) .. _exec-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/Processes%2Fpaths%2F~1processes~1{process_id}~1jobs%2Fpost .. |vis-req| replace:: ``PUT {WEAVER_URL}/processes/{id}/visibility`` (Visibility) diff --git a/docs/source/toc.rst b/docs/source/toc.rst index 831a0fefe..2c01da497 100644 --- a/docs/source/toc.rst +++ b/docs/source/toc.rst @@ -12,6 +12,7 @@ Table of Contents tutorial processes package + Examples OpenAPI Specification Source Code Documentation changes diff --git a/weaver/__meta__.py b/weaver/__meta__.py index 627d92bed..22ec71067 100644 --- a/weaver/__meta__.py +++ b/weaver/__meta__.py @@ -2,8 +2,6 @@ __title__ = "Weaver" __version__ = "1.13.1" __description__ = "Workflow Execution Management Service (EMS); Application, Deployment and Execution Service (ADES)." -# __source_repository__ = "https://www.crim.ca/stash/projects/OGC/repos/weaver" -# __docker_repository__ = "docker-registry.crim.ca/ogc/weaver:latest" __source_repository__ = "https://github.com/crim-ca/weaver" __docker_repository__ = "https://hub.docker.com/r/pavics/weaver" __license_type__ = "Apache License 2.0" From 4718656ab135062790b0fc60e23d3dcb6003e23b Mon Sep 17 00:00:00 2001 From: Francis Charette Migneault Date: Thu, 24 Sep 2020 16:07:33 -0400 Subject: [PATCH 2/3] patch requested build doc formats --- .readthedocs.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.readthedocs.yml b/.readthedocs.yml index f1ca28b64..71e2f34c7 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -2,7 +2,9 @@ version: 2 sphinx: configuration: docs/source/conf.py -formats: all +#formats: all +formats: + - htmlzip python: version: 3.7 install: From c6635d8dcc6639dcaf85328b7bbce32d728c16ac Mon Sep 17 00:00:00 2001 From: Francis Charette Migneault Date: Fri, 25 Sep 2020 12:30:33 -0400 Subject: [PATCH 3/3] fix doc8 reference flagged --- docs/source/examples.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/examples.rst b/docs/source/examples.rst index 11398f824..b27d40917 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -1,4 +1,4 @@ -.. _examples: +.. _examples-page: .. include:: references.rst ********************************************