Sc1 demo

eo/CHANGELOG.md

@@ -3,18 +3,18 @@ Changes
 
 Unreleased (latest)
 ===================
-0.3.1 (2023-11-22)
+0.4.1 (2023-11-22)
 ===================
 Changes:
 --------
 - Update base image version in Dockerfile
 - Add `mamba` cache cleanup to reduce image size
-- Add Sc1-Demontrator notebook of a complete workflow as example for spatial imagerie analysis and process execution
+- Add SC1-Demontrator notebook of a complete workflow as example for geo imagery analysis and OGC-API Process execution
 - Add Process Management notebook to help user understand the usage of weaver and how to manage deployed process
 
 Fixes:
 ------
-- ...
+- n/a
 
 0.3.0 (2022-12-23)
 ===================

eo/notebooks/SC1 Demonstrator - Multi-Temporal Analysis on a Sentinel-1 Datacube.ipynb → eo/notebooks/SC1-Demonstrator - Geo Spatial Imagery Processing And Analysis.ipynb

@@ -6,7 +6,7 @@
    "metadata": {},
    "source": [
     "# SC1 Demonstrator \n",
-    "## Geo Spatial Imagery Processing And Analysis \n",
+    "## Geo Imagery Processing And Analysis Workflow Using Daccs Infrastructure\n",
     "\n",
     "This notebook serves as a demonstrator of a complete workflow for of Sentinel Geo Spatial imagery processing and analysis."
    ]
@@ -44,9 +44,17 @@
    "metadata": {},
    "source": [
     "---\n",
-    "**NOTE**\n",
+    "### Requirements\n",
+    "\n",
+    "Processes `./demo/sentinelsat-application.cwl` and `./demo/eo-tools-application.cwl` should be deployed on the machine. </br>\n",
+    "\n",
+    "If they are not, follow the section : `2.Deploy the process on weaver` from the `./Process_Management_Demonstration` notebook.\n",
+    "\n",
+    "\n",
+    "---\n",
+    "**NOTES**\n",
     "\n",
-    "This demonstrator notebook relies on the availability of certain online products and heavy image processing. Because of the long download and rendering of the processes, some sections of this demo are not supposed to work and are there only as an example. \n",
+    "This demonstrator notebook relies on the availability of certain online products and heavy image processing. These products are usually archived after a long time of none usage from the users. Because of that these sectionsthey can be used as examples. \n",
     "\n",
     "The concerned sections: \n",
     "* 2.5 [Product Download With Weaver](#2.5-Product-Download-With-Weaver)\n",
@@ -125,7 +133,6 @@
        " 'file2string_array',\n",
        " 'jsonarray2netcdf',\n",
        " 'metalink2netcdf',\n",
-       " 'sentinelsat_batch_download',\n",
        " 'sentinelsat_download']"
       ]
      },
@@ -136,7 +143,7 @@
    ],
    "source": [
     "# # Getting the available processes and validating that we have the processes needed: \n",
-    "# 1. sentinelsat_batch_download \n",
+    "# 1. sentinelsat_download \n",
     "# 2. eo_tools\n",
     "wclient.capabilities().body['processes']"
    ]
@@ -167,7 +174,7 @@
    "source": [
     "### Documentation\n",
     "\n",
-    "For the users willing to use the SentinelSat librarie:\n",
+    "For the users willing to use the SentinelSat library:\n",
     "- [Sentinelsat](https://sentinelsat.readthedocs.io/en/stable/)<br/>\n"
    ]
   },
@@ -572,7 +579,7 @@
    "id": "80d20d97",
    "metadata": {},
    "source": [
-    "![sentinelsat_products_map](images/sentinelsat_products_map.png)"
+    "![sentinelsat_products_map](demo/sentinelsat_products_map.png)"
    ]
   },
   {
@@ -692,14 +699,14 @@
    "metadata": {},
    "source": [
     "## 2.5 Product Download With Weaver \n",
-    "For the execution, Weaver expect a payload containing the inputs.\n",
+    "For the execution, Weaver expects a payload containing the inputs.\n",
     "Because our process is a downloader of Spatial sentinel images we will need two inputs:\n",
     "1. <b> The credentials file</b> \n",
     "\n",
     "2. <b>The name of the image to be downloaded</b>\n",
     "\n",
     "* **Important note** </br>\n",
-    "Because copernicus archives the product, if it has not been used or downloaded in a long time and therfor the product is archived, our SentinelSat download process might finish successfully without downloading the file. That is because Weaver does not have a retry on fail. So for this example we will use an online product. "
+    "Because copernicus archives the product, if it has not been used or downloaded in a long time and therefore the product is archived, our SentinelSat download process might finish successfully without downloading the file. That is because Weaver does not have a retry on fail. So for this example we will use an online product. "
    ]
   },
   {
@@ -729,7 +736,7 @@
     }
    ],
    "source": [
-    "# Defining the credentials for the copernicus api used by the sentinelsat_batch_download process \n",
+    "# Defining the credentials for the copernicus api used by the sentinelsat_download process \n",
     "print(\" Enter your copernicus credentials here :\")\n",
     "credentials = {\n",
     "    \"username\": input(\"\\tUsername: \"),\n",
@@ -768,7 +775,7 @@
    "outputs": [],
    "source": [
     "# Defining the dictionary or json to be sent to weaver to be used as inputs for the process to be executed \n",
-    "sentinelsat_batch_download_inputs = {\n",
+    "sentinelsat_download_inputs = {\n",
     "  \"mode\": \"async\",\n",
     "  \"response\": \"document\",\n",
     "  \"inputs\": [\n",
@@ -803,7 +810,7 @@
    "outputs": [],
    "source": [
     "# Executing the download process trought weaver\n",
-    "execute_response = wclient.execute(process_id=\"sentinelsat_batch_download\", inputs=sentinelsat_batch_download_inputs)\n",
+    "execute_response = wclient.execute(process_id=\"sentinelsat_download\", inputs=sentinelsat_download_inputs)\n",
     "\n",
     "# This will delete de temporary credentials file\n",
     "credentials_file.close()"
@@ -844,7 +851,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "processID : sentinelsat_batch_download\n",
+      "processID : sentinelsat_download\n",
       "status :    succeeded\n",
       "progress :  100\n",
       "duration :  00:00:36\n"

eo/notebooks/demo/eo-tools-application.cwl

@@ -0,0 +1,65 @@
+#!/usr/bin/env cwl-runner
+
+class: CommandLineTool
+cwlVersion: v1.0
+
+label: "Application to process Sentinel products ESA's Snap software thought a python wrapper."
+
+requirements:
+  DockerRequirement:
+    dockerPull: registry.gitlab.com/crim.ca/public/daccs/daccs/daccs-eo-tools:0.1.1
+inputs:
+  command:
+    type: string
+    inputBinding:
+      position: 1
+  source:
+    type: File[]
+    inputBinding:
+      position: 2
+      prefix: --source
+  output_name:
+    type: string?
+    inputBinding:
+      position: 3
+      prefix: --output
+  dem:
+    type: string?
+    inputBinding:
+      position: 4
+      prefix: --dem
+  polarization:
+    type: string?
+    inputBinding:
+      position: 5
+      prefix: --polarization
+  subswath:
+    type: string?
+    inputBinding:
+      position: 6
+      prefix: --subswath
+  refband:
+    type: string?
+    inputBinding:
+      position: 7
+      prefix: --refband
+
+outputs:
+  output:
+    type: File
+    outputBinding:
+      glob: $(runtime.outdir)/*.nc
+
+# Metadata is not visible on Weaver once application is deployed; this is mainly to conserve the information
+s:author:
+  - class: s:Person
+    s:email: [email protected]
+    s:name: Francis Pelletier
+# Update version manually after changes to CWL application changes
+s:version: 0.1.0
+
+$namespaces:
+  s: https://schema.org/
+
+$schemas:
+  https://schema.org/version/latest/schemaorg-current-http.rdf

eo/notebooks/demo/sentinelsat-application.cwl

@@ -0,0 +1,50 @@
+#!/usr/bin/env cwl-runner
+
+# Changes to this application should also be mirrored in config/sentinelsat/applications/sentinelsat-application.cwl
+class: CommandLineTool
+cwlVersion: v1.0
+
+label: "Application to batch download Sentinel products."
+
+
+requirements:
+  DockerRequirement:
+    dockerPull: registry.gitlab.com/crim.ca/public/daccs/daccs/daccs-eo-sentinelsat:0.2.1
+arguments: [ "-d" ]
+inputs:
+  credentials:
+    type: File
+    inputBinding:
+      position: 1
+      prefix: --credentials
+  image:
+    type: string[]
+    inputBinding:
+      prefix: --name
+      itemSeparator: ","
+      position: 2
+
+outputs:
+  download_status:
+    type: File
+    outputBinding:
+      glob: $(runtime.outdir)/download_succeeded.txt
+  output:
+    type: Directory
+    outputBinding:
+      glob: $(runtime.outdir)
+
+# Metadata is not visible on Weaver once application is deployed; this is mainly to conserve the information
+s:author:
+  - class: s:Person
+    s:email: [email protected]
+    s:name: Francis Pelletier
+# Update version manually after changes to CWL application changes
+s:version: 0.1.0
+
+$namespaces:
+  s: https://schema.org/
+
+$schemas:
+  https://schema.org/version/latest/schemaorg-current-http.rdf
+