chore(docs): update provider plugin docs (#2535)

Author: rokroskar

docs/how-to-guides/hpc.rst

@@ -3,10 +3,12 @@
 Running Renku on HPC
 ====================
 
-Renku CLI supports various backends for executing workflows. Currently, there
-are two different providers are implemented, namely ``cwltool`` and ``toil``.
-:ref:`provider` documents gives a more detailed description of how to implement
-your own workflow provider.
+The Renku CLI supports various backends for executing workflows. Currently, two
+different providers are implemented, namely ``cwltool`` and ``toil``. If you
+have a specific provider you need for your infrastructure, have a look at
+:doc:`implementing_a_provider` for a detailed description of how to implement
+your own workflow provider. Alternatively, please `make a feature request
+<https://github.com/SwissDataScienceCenter/renku-python/issues/new?assignees=&labels=&template=feature_request.md>`_.
 
 By default all workflows are executed by the ``cwltool`` provider, that
 exports the workflow to CWL and then uses `cwltool <https://github.com/common-workflow-language/cwltool>`_
@@ -18,12 +20,12 @@ providing the  ``-c/--config <config.yaml>`` command line parameter.
 The following ``renku`` commands support the above mentioned workflow provider
 related command line options:
 
- - :ref:`cli-rerun`,
- - :ref:`cli-update`,
- - :ref:`cli-workflow` ``execute`` and ``iterate``.
+ - :ref:`cli-rerun`
+ - :ref:`cli-update`
+ - :ref:`cli-workflow` ``execute`` and ``iterate``
 
 For example, to execute a previously created ``my_plan`` workflow with ``toil``, one
-simply would run the following command:
+would simply run the following command:
 
 .. code-block:: console
 
@@ -34,23 +36,34 @@ the workflows on various `high-performance computing <https://toil.readthedocs.i
 and `cloud <https://toil.readthedocs.io/en/latest/running/cloud/cloud.html#cloud-platforms>`_
 platforms.
 
+In order to use any other provider with ``renku`` you must first install the required
+extras. In the case of ``toil`` this means running the install command like
+
+.. code-block:: console
+
+   $ pip install renku[toil]
+
+
 Renku on Slurm
 ^^^^^^^^^^^^^^
-`Slurm <https://www.schedmd.com/>`_ is a highly configurable open-source workload manager,
-which is in widespread use at government laboratories, universities and companies world
-wide and performs workload management for over half of the top 10 systems in the TOP500.
+
+`Slurm <https://www.schedmd.com/>`_ is a highly configurable open-source
+workload manager, which is in widespread use at government laboratories,
+universities and companies world wide. It is used in over half of the top 10
+systems in the `TOP500 <https://www.top500.org/>`_ listing.
 
 As ``toil`` supports Slurm, one can easily execute the previously created renku
-workflows on Slurm. One just needs to provide a simple configuration file to the provider
-(``--config``)::
+workflows on a Slurm-managed HPC resource. One just needs to provide a simple
+configuration file to the provider (``--config/-c``)::
 
   batchSystem: slurm
   disableCaching: true
 
-The ``disableCaching`` is necessary to be enabled for Slurm, for more details see the
-related ``toil`` issue `TOIL-1006 <https://ucsc-cgl.atlassian.net/browse/TOIL-1006>`_.
+The ``disableCaching`` option is necessary for Slurm; for more details see the
+related ``toil`` issue `TOIL-1006
+<https://ucsc-cgl.atlassian.net/browse/TOIL-1006>`_.
 
-`Additional Slurm specific parameters <https://slurm.schedmd.com/sbatch.html>`_ can be
+`Additional Slurm-specific parameters <https://slurm.schedmd.com/sbatch.html>`_ can be
 provided with the ``TOIL_SLURM_ARGS`` environment variable.
 
 Taking the example above, the following command line will execute ``my_plan`` on Slurm:

docs/how-to-guides/provider.rst …ow-to-guides/implementing_a_provider.rstdocs/how-to-guides/provider.rst renamed to docs/how-to-guides/implementing_a_provider.rst docs/how-to-guides/provider.rst renamed to docs/how-to-guides/implementing_a_provider.rst

@@ -1,12 +1,11 @@
-.. _provider:
+.. _implementing_a_provider:
 
 Implementing a workflow provider
 ================================
 
-In the previous section about :ref:`hpc`, we described how using different
-workflow providers can enable the user running renku workflows on HPC. In
-this short article we will discuss how to implement our own, custom workflow
-provider as a plugin for Renku CLI.
+In  :ref:`hpc`, we described how using different workflow providers can enable
+the user running renku workflows on HPC. Here we discuss how to implement a new,
+custom workflow provider as a plugin for Renku CLI.
 
 Renku provides the option to add a new workflow executor backend with the
 help of `pluggy <https://pluggy.readthedocs.io/en/latest/>`_ plugins.

docs/how-to-guides/index.rst

@@ -8,9 +8,8 @@ aimed at active users of Renku CLI and target specific use-cases or common issue
 
 
 .. toctree::
-   :maxdepth: 2
-
+   :maxdepth: 1
 
    hpc
-   provider
+   implementing_a_provider
    shell-integration

docs/reference/plugins.rst

@@ -81,3 +81,13 @@ where `myproject.pluginmodule:mycmd` points to a click command e.g.:
     @click.command()
     def mycmd():
         ...
+
+Workflow Provider Plugins
+-------------------------
+
+Additional workflow providers can be implemented by extending
+:class:`renku.core.models.workflow.provider.IWorkflowProvider`. See
+:ref:`implementing_a_provider` for more information.
+
+.. autoclass:: renku.core.models.workflow.provider.IWorkflowProvider
+    :members: