Expand documentation

docs/index.md

@@ -20,7 +20,7 @@ links to further resources.
 2. [Installing xcengine](installing.md)
 3. [Making a Jupyter notebook xcengine-compatible](notebook.md)
 4. [Running `xcetool`](xcetool.md)
-5. [Testing and running Application Packages](testing.md)
+5. [Testing and running Application Packages](running-eoaps.md)
 6. [Further documentation and resources](resources.md)
 
 xcengine is developed as part of the [Open Earth Monitor

docs/intro.md

@@ -24,10 +24,10 @@ An Application Package consists of two parts:
    which defines how the code in the container image should be run,
    and what its available parameters, inputs, and outputs are.
 
-Application Packages are designed to be run on cloud processing
-platforms, but can also be tested locally. Application Packages provide
-a great deal of power and flexibility; partly because of this flexibility, they
-can be complex and challenging to build from scratch.
+Application Packages are designed to be run on cloud processing platforms, but
+can also be tested locally. Application Packages provide a great deal of power
+and flexibility; partly because of this flexibility, they can be complex and
+challenging to build from scratch.
 
 ## xcube Server and Viewer
 

docs/resources.md

@@ -4,22 +4,22 @@ xcengine has been presented at several conferences and workshops. Most of the
 presentation materials are available online and provide a useful complement to
 this documentation.
 
-Presentation at OEMC Global Workshop 2024, Laxenburg.
+- Presentation at OEMC Global Workshop 2024, Laxenburg.
 Pontus Lurcock, "Large-scale EO processing with xcube on CDSE". 
 [Video recording of presentation](https://www.youtube.com/watch?v=e0P1KVyvoEc&list=PLXUoTpMa_9s3g9lD7_aP0XB12wluSPhRJ&index=28).
 
-Presentation at ESA Living Planet Symposium 2025, Vienna.
+- Presentation at ESA Living Planet Symposium 2025, Vienna.
 Pontus Lurcock, Tejas Morbagal Harish, and Alicja Balfanz,
 "From notebooks to application packages with xcube and xcengine".
 [Slide deck](https://lps25.esa.int/lps25-presentations/presentations/1276/lurcock_update.pdf).
 
-Tutorial at OpenGeoHub EO-Council Summer School 2025, Wageningen:
+- Tutorial at OpenGeoHub EO-Council Summer School 2025, Wageningen:
 "Application packages in the xcube ecosystem".
 [Course page](https://xcube-dev.github.io/summerschool25/) with links to
 slides, documentation, and hands-on exercises.
 [Video recording of presentation](https://www.youtube.com/watch?v=b3BdOE3mJZU).
 
-ESA Big Data from Space 2025, Riga.
+- Presentation at ESA Big Data from Space 2025, Riga.
 Yogesh Baljeet Singh, Norman Fomferra, Pontus Lurcock, Gunnar Brandt and
 Tejas Morbagal Harish: "From reproducible EO Workflow development to OGC
 Application Packages for Scalable Data Cube Generation and Visualisation".

docs/testing.md → docs/running-eoaps.md

@@ -1,14 +1,15 @@
 # Testing and running Application Packages
 
-Application Packages are generally deployed to cloud platforms, but there are also
-ways to run CWL files and complete Application Packages locally.
+Application Packages are generally deployed to cloud platforms, but there are
+also ways to run CWL files and complete Application Packages locally.
 
 ## Running with cwltool
 
-[cwltool](https://www.commonwl.org/user_guide/introduction/quick-start.html) is the
-reference runner for CWL files. It doesn't implement the full Application Package
-best practice (so there is no stage-in / stage-out functionality) but can nevertheless
-be used to run CWL files that implement Application Packages.
+[cwltool](https://www.commonwl.org/user_guide/introduction/quick-start.html)
+is the reference runner for CWL files. It doesn't implement the full
+Application Package best practice (so there is no stage-in / stage-out
+functionality) but can nevertheless be used to run CWL files that implement
+Application Packages.
 
 ## Running with ZOO-Project and the DRU extensions
 
@@ -18,3 +19,10 @@ use the [ZOO-Project](https://zoo-project.org/) with the optional
 [minikube](https://minikube.sigs.k8s.io/docs/). The
 [EOEPCA+ documentation](https://eoepca.readthedocs.io/projects/deploy/en/2.0-rc1/building-blocks/oapip-engine/)
 gives detailed instructions for installation.
+
+## Deploying your Application Package to a platform
+
+Once your container image has been pushed to a registry, the CWL file can be
+deployed to a cloud platform to make your Application Package available to
+users of this platform. The process for deploying the CWL file varies; consult
+the platform documentation or support for details.

docs/xcetool.md

@@ -11,48 +11,97 @@ details on usage and available options.
 
 ### `xcetool image build`
 
+Usage: `xcetool image build [OPTIONS] NOTEBOOK`
+
 This is the main `xcetool` subcommand: it builds a container image from a supplied
 notebook and environment file. If given the `--eoap` argument, it also generates
 a CWL file defining a corresponding application package.
 
+Options:
+
+-   `-b`, `--build-dir` `DIRECTORY`: Build directory to use for preparing the
+    Docker image. If not specified, an automatically created temporary
+    directory will be used.
+    This option is mainly useful for debugging.
+-   `-e`, `--environment` `FILE`:
+    Conda environment file to use in Docker image.
+    If no environment file is specified here or in the notebook,
+    xcetool will try to reproduce the current environment
+    as a last resort, but this is not recommended.
+-   `-t`, `--tag` `TEXT`: Tag to apply to the Docker image.
+    If not specified, a
+    timestamp-based tag will be generated automatically.
+-   `-a`, `--eoap` `PATH`: Write a CWL file defining an Earth Observation
+    Application Package to the specified path.
+-   `--help`: Show a help message for this subcommand and exit.
+
 ### `xcetool image run`
 
-This subcommand runs an xcengine container image. An image can also be run using the
-`docker run` command, but `xcetool image run` provides some additional convenience
-(e.g. easy configuration of a server HTTP port).
+Usage: `xcetool image run [OPTIONS] IMAGE`
+
+Options:
+
+-   `-b`, `--batch`: Run the compute engine as a batch script.
+    Use with the `--output` option to copy output
+    out of the container.
+-   `-s`, `--server`: Run the compute engine as an xcube server.
+-   `-p`, `--port` INTEGER: Host port for xcube server (default: 8080).
+    Implies `--server`.
+-   `-f`, `--from-saved`: If `--batch` and `--server` both used, serve
+    datasets from saved Zarrs rather than
+    computing them on the fly.
+-   `-o`, `--output` DIRECTORY  Write any output data to this directory,
+    which will be created if it does not exist
+    already.
+-   `-k`, `--keep`: Keep container after it has finished
+    running.
+-   `--help`: Show a help message for this subcommand and exit.
+
+This subcommand runs an xcengine container image. An image can also be run
+using the `docker run` command, but `xcetool image run` provides some
+additional convenience (e.g. easy configuration of a server HTTP port).
+
+If you use the `--server` option with `xcetool image run`, the image will be
+run in xcube server mode: after the code from the input notebook is used to
+generate datasets, those datasets will be made available in an xcube server
+instance. You can also use the `--port` option to select the HTTP port where
+the xcube server should be exposed. The server also includes an interactive
+web viewer component. On start-up, `xcetool` will print the URLs of the xcube
+server and viewer to the standard output.
+
+If you give the `--server` or `--port` options, `xcetool` will run the
+container indefinitely as an xcube server and viewer instance. You can stop
+the container and force `xcetool` to exit by pressing ctrl-C on the command
+line (or by sending it an interrupt signal in some other way).
 
-If you use the `--server` option with `xcetool image run`, the image will be run in
-xcube server mode: after the code from the input notebook is used to generate datasets,
-those datasets will be made available in an xcube server instance. You can also use
-the `--port` option to select the HTTP port where the xcube server should be exposed.
-The server also includes an interactive web viewer component. On start-up, `xcetool`
-will print the URLs of the xcube server and viewer to the standard output. 
+### `xcetool make-script` 
 
-If you give the `--server` or `--port` options, `xcetool` will run the container
-indefinitely as an xcube server and viewer instance. You can stop the container and
-force `xcetool` to exit by pressing ctrl-C on the command line (or by sending it an
-interrupt signal in some other way).
+This subcommand does not generate a container image, but a directory
+containing the main contents that the image would have had: a script derived
+from the notebook along with some supporting files. The `make-script`
+subcommand is mainly useful for debugging.
 
-### `xcetool make-script` 
+Usage: `xcetool make-script [OPTIONS] NOTEBOOK OUTPUT_DIR`
+
+Create a compute engine script on the host system. The output directory will
+be used for the generated script, supporting code modules, and any output
+produced by running the script.
 
-This subcommand does not generate a container image, but a directory containing
-the main contents that the image would have had: a script derived from the notebook
-along with some supporting files. The `make-script` subcommand is mainly useful for
-debugging.
+Options:
+
+-   `-b`, `--batch`: Run as batch script after creating
+-   `-s`, `--server`: Run the script as an xcube server after creating it.
+-   `-f`, `--from-saved`: If `--batch` and `--server` both used, serve datasets from saved Zarrs rather than computing them on the fly.
+-   `-c`, `--clear`: Clear output directory before writing to it
+-   `--help`: Show a help message for this subcommand and exit.
 
 ## Publishing your container image
 
 Once you have built your container image locally, you can push it to an online
 registry. The tag is used to determine the registry and repository to which to
 push the image. For example, if an image is tagged `quay.io/alice/helloworld:1.0`,
 pushing it will attempt to upload the image to the repository `helloworld`
-under the account `alice` on the registry `quay.io`.
-
-
-## Deploying your Application Package
-
-Once your container image has been pushed to a registry, the CWL file can be deployed
-to a cloud platform to make your Application Package available to users of this
-platform. The process for deploying the CWL file varies; consult the platform
-documentation or support for details.
+under the account `alice` on the registry `quay.io`. See the [Docker
+documentation](https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-registry/)
+for more details.
 

mkdocs.yml

@@ -2,3 +2,11 @@ site_name: xcengine
 site_url: https://xcube-dev.github.io/xcengine
 theme:
   name: material
+nav:
+  - Home: index.md
+  - Introduction: intro.md
+  - Installation: installing.md
+  - Preparing your notebook: notebook.md
+  - Running xcengine: xcetool.md
+  - Running Application Packages: running-eoaps.md
+  - Further resources: resources.md