Geoserver: protect web interface and ows routes behind magpie/twitcher (

#348)

## Overview

Geoserver: protect web interface and ows routes behind magpie/twitcher
  
Updates Magpie version to
[3.35.0](https://github.com/Ouranosinc/Magpie/tree/3.35.0) in order to
take advantage of updated Geoserver Service.

The `geoserverwms` Magpie service is now deprecated. If a deployment is
currently using this service, it is highly recommended that the
permissions are transferred from the deprecated `geoserverwms` service
to the `geoserver` service.

The `/geoserver` endpoint is now protected by default. If a deployment
currently assumes open access to Geoserver and would like to keep the
same permissions after upgrading to this version, please update the
permissions for the `geoserver` service in Magpie to allow the
`anonymous` group access.

A `Magpie` service named `geoserver` with type `wfs` exists already and
must be manually deleted before the new `Magpie` service created here
can take effect.

The `optional-components/all-public-access` component provides full
access to the `geoserver` service for the `anonymous` group in Magpie.
Please note that this includes some permissions that will allow
anonymous users to perform destructive operations. Because of this,
please remember that enabling the
`optional-components/all-public-access` component is not recommended in
a production environment.

Introduces the `GEOSERVER_SKIP_AUTH` environment variable. If set to
`True`, then requests to the geoserver endpoint will not be authorized
through twitcher/magpie at all. This is not recommended at all. However,
it will slightly improve performance when accessing geoserver endpoints.

## Changes

**Non-breaking changes**

In order to provide public access to geoserver by default now, the
`all-public-access` optional component must be enabled

**Breaking changes**

The current `wfs` Magpie service named `geoserver` must be deleted
before the change here can take effect.

## Related Issue / Discussion

- Resolves #333

## Additional Information

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.36.0
+current_version = 1.37.0
 commit = True
 tag = False
 tag_name = {new_version}
@@ -30,11 +30,11 @@ search = {current_version}
 replace = {new_version}
 
 [bumpversion:file:RELEASE.txt]
-search = {current_version} 2023-10-31T17:20:38Z
+search = {current_version} 2023-11-01T13:09:36Z
 replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ}
 
 [bumpversion:part:releaseTime]
-values = 2023-10-31T17:20:38Z
+values = 2023-11-01T13:09:36Z
 
 [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template]
 search = 'version': '{current_version}'

CHANGES.md

@@ -17,6 +17,37 @@
 
 [//]: # (list changes here, using '-' for each new entry, remove this when items are added)
 
+[1.37.0](https://github.com/bird-house/birdhouse-deploy/tree/1.37.0) (2023-11-01)
+------------------------------------------------------------------------------------------------------------------
+
+## Changes
+- Geoserver: protect web interface and ows routes behind magpie/twitcher
+
+  Updates Magpie version to [3.35.0](https://github.com/Ouranosinc/Magpie/tree/3.35.0) in order to take advantage of 
+  updated Geoserver Service.
+
+  The `geoserverwms` Magpie service is now deprecated. If a deployment is currently using this service, it is highly
+  recommended that the permissions are transferred from the deprecated `geoserverwms` service to the `geoserver` 
+  service.
+
+  The `/geoserver` endpoint is now protected by default. If a deployment currently assumes open access to Geoserver and 
+  would like to keep the same permissions after upgrading to this version, please update the permissions for the 
+  `geoserver` service in Magpie to allow the `anonymous` group access.
+
+  A `Magpie` service named `geoserver` with type `wfs` exists already and must be manually deleted before the new
+  `Magpie` service created here can take effect.
+
+  The `optional-components/all-public-access` component provides full access to the `geoserver` service for the 
+  `anonymous` group in Magpie. Please note that this includes some permissions that will allow anonymous users to 
+  perform destructive operations. Because of this, please remember that enabling the 
+  `optional-components/all-public-access` component is not recommended in a production environment.
+
+  Introduces the `GEOSERVER_SKIP_AUTH` environment variable. If set to `True`, then requests to the geoserver endpoint 
+  will not be authorized through twitcher/magpie at all. This is not recommended at all. However, it will slightly 
+  improve performance when accessing geoserver endpoints.
+
+  See https://github.com/bird-house/birdhouse-deploy/issues/333 for details.
+
 [1.36.0](https://github.com/bird-house/birdhouse-deploy/tree/1.36.0) (2023-10-31)
 ------------------------------------------------------------------------------------------------------------------
 

Makefile

@@ -1,7 +1,7 @@
 # Generic variables
 override SHELL       := bash
 override APP_NAME    := birdhouse-deploy
-override APP_VERSION := 1.36.0
+override APP_VERSION := 1.37.0
 
 # utility to remove comments after value of an option variable
 override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")

README.rst

@@ -14,13 +14,13 @@ for a full-fledged production platform.
     * - releases
       - | |latest-version| |commits-since|
 
-.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.36.0.svg
+.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.37.0.svg
     :alt: Commits since latest release
-    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.36.0...master
+    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.37.0...master
 
-.. |latest-version| image:: https://img.shields.io/badge/tag-1.36.0-blue.svg?style=flat
+.. |latest-version| image:: https://img.shields.io/badge/tag-1.37.0-blue.svg?style=flat
     :alt: Latest Tag
-    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.36.0
+    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.37.0
 
 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
     :alt: ReadTheDocs Build Status (latest version)

RELEASE.txt

@@ -1 +1 @@
-1.36.0 2023-10-31T17:20:38Z
+1.37.0 2023-11-01T13:09:36Z

birdhouse/config/canarie-api/docker_configuration.py.template

@@ -109,8 +109,8 @@ SERVICES = {
             # NOTE:
             #   Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
             #   Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
-            'version': '1.36.0',
-            'releaseTime': '2023-10-31T17:20:38Z',
+            'version': '1.37.0',
+            'releaseTime': '2023-11-01T13:09:36Z',
             'institution': 'Ouranos',
             'researchSubject': 'Climatology',
             'supportEmail': '${SUPPORT_EMAIL}',
@@ -142,8 +142,8 @@ PLATFORMS = {
             # NOTE:
             #   Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
             #   Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
-            'version': '1.36.0',
-            'releaseTime': '2023-10-31T17:20:38Z',
+            'version': '1.37.0',
+            'releaseTime': '2023-11-01T13:09:36Z',
             'institution': 'Ouranos',
             'researchSubject': 'Climatology',
             'supportEmail': '${SUPPORT_EMAIL}',

birdhouse/config/geoserver/.gitignore

@@ -1,9 +1,9 @@
 config/proxy/conf.extra-service.d/geoserver.conf
 config/canarie-api/canarie_api_monitoring.py
-config/magpie/providers.cfg
 service-config.json
 
 # Old paths. Keep these so that old config files remain uncommittable after updates.
 geoserver_canarie_api_monitoring.py
 geoserver-magpie-provider.cfg
 config/proxy/canarie_api_monitoring.py
+config/magpie/providers.cfg

birdhouse/config/geoserver/config/magpie/providers.cfg

@@ -0,0 +1,12 @@
+providers:
+  geoserver:
+    # below URL is only used to fill in the required location in Magpie
+    # actual auth validation is performed with Twitcher 'verify' endpoint without accessing this proxied URL
+    url: http://proxy:80
+    title: geoserver
+    type: geoserver
+    configuration:
+      wfs: true
+      wms: true
+      wps: false
+      api: true

birdhouse/config/geoserver/config/proxy/conf.extra-service.d/geoserver.conf.template

@@ -1,4 +1,7 @@
     location /geoserver/ {
+        auth_request /secure-geoserver-auth;
+        auth_request_set $auth_status $upstream_status;
+
         proxy_pass http://geoserver:8080/geoserver/;
         proxy_set_header Host $host;
         proxy_set_header X-Forwarded-Proto $real_scheme;
@@ -8,3 +11,20 @@
         gzip_comp_level 1;
         gzip_types application/json text/plain application/xml text/html;
     }
+
+    location = /secure-geoserver-auth {
+        internal;
+        ${GEOSERVER_SKIP_AUTH_PROXY_INCLUDE}
+
+        # If GEOSERVER_SKIP_AUTH is "True" then the following section is skipped and this
+        # location block will always return 200 (which means that the /geoserver/ location, above,
+        # will be publicly available.
+        proxy_pass https://${PAVICS_FQDN_PUBLIC}${TWITCHER_VERIFY_PATH}/geoserver$request_uri;
+        proxy_pass_request_body off;
+        proxy_set_header Host $host;
+        proxy_set_header Content-Length "";
+        proxy_set_header X-Original-URI $request_uri;
+        proxy_set_header X-Forwarded-Proto $real_scheme;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Host $host:$server_port;
+    }

birdhouse/config/geoserver/default.env

@@ -26,9 +26,14 @@ export GEOSERVER_COMMUNITY_EXTENSIONS="geopkg-plugin"
 # Must use single-quote for delayed eval.
 export GEOSERVER_DATA_DIR='${DATA_PERSIST_ROOT}/geoserver'
 
+# If set, requests to the geoserver endpoint will not be authorized through twitcher/magpie
+export GEOSERVER_SKIP_AUTH=False
+export GEOSERVER_SKIP_AUTH_PROXY_INCLUDE='$([ x"${GEOSERVER_SKIP_AUTH}" = x"True" ] && echo "return 200;")'
+
 export DELAYED_EVAL="
   $DELAYED_EVAL
   GEOSERVER_DATA_DIR
+  GEOSERVER_SKIP_AUTH_PROXY_INCLUDE
 "
 
 # add any new variables not already in 'VARS' or 'OPTIONAL_VARS' that must be replaced in templates here
@@ -44,4 +49,5 @@ OPTIONAL_VARS="
   \$GEOSERVER_VERSION
   \$GEOSERVER_TAGGED
   \$GEOSERVER_IMAGE
+  \$GEOSERVER_SKIP_AUTH_PROXY_INCLUDE
 "

birdhouse/config/magpie/default.env

@@ -5,7 +5,7 @@
 # are applied and must be added to the list of DELAYED_EVAL.
 
 # Tag version that will be used to update Magpie API, Magpie CLI, and matching Twitcher with Magpie Adapter
-export MAGPIE_VERSION=3.34.0
+export MAGPIE_VERSION=3.36.0
 
 export MAGPIE_DB_NAME="magpiedb"
 

birdhouse/env.local.example

@@ -484,6 +484,9 @@ export GEOSERVER_ADMIN_PASSWORD=geoserverpass
 # (note: if using 'DATA_PERSIST_ROOT', it must be defined earlier, either in this file or from 'default.env')
 #export WEAVER_MONGODB_DATA_DIR='${DATA_PERSIST_ROOT}/mongodb_weaver_persist'
 
+# If "True", requests to the geoserver endpoint will not be authorized through twitcher/magpie
+# (note: this is NOT recommended but will slightly improve performance when accessing geoserver endpoints)
+#export GEOSERVER_SKIP_AUTH=True
 
 #############################################################################
 # Monitoring components configs

birdhouse/optional-components/all-public-access/config/geoserver/docker-compose-extra.yml

@@ -0,0 +1,5 @@
+version: "3.4"
+services:
+  magpie:
+    volumes:
+      - ./optional-components/all-public-access/config/geoserver/permissions.cfg:${MAGPIE_PERMISSIONS_CONFIG_PATH}/all-public-access-geoserver-permissions.cfg:ro

birdhouse/optional-components/all-public-access/config/geoserver/permissions.cfg

@@ -0,0 +1,70 @@
+permissions:
+ - service: geoserver
+   type: route
+   permission: read
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: describestoredqueries
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: describelayer
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: dropstoredquery
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: lockfeature
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getmap
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getfeature
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getfeaturewithlock
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getfeatureinfo
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getgmlobject
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getpropertyvalue
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: transaction
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: createstoredquery
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getlegendgraphic
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: getcapabilities
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: describefeaturetype
+   group: anonymous
+   action: create
+ - service: geoserver
+   permission: liststoredqueries
+   group: anonymous
+   action: create

docs/source/conf.py

@@ -69,9 +69,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.36.0'
+version = '1.37.0'
 # The full version, including alpha/beta/rc tags.
-release = '1.36.0'
+release = '1.37.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.