feat: Support removing properties from catalogs (#713)

* feat: Support removing properties from catalogs

* changelog

* Add docs

* Improve docs

* fix merge conflicts

* Apply suggestions from code review

Co-authored-by: Malte Sander <[email protected]>

* charts

---------

Co-authored-by: Malte Sander <[email protected]>

Author: sbernauer

CHANGELOG.md

@@ -12,6 +12,8 @@ All notable changes to this project will be documented in this file.
 - Support configuring JVM arguments ([#677]).
 - Aggregate emitted Kubernetes events on the CustomResources ([#677]).
 - Support for Trino 470 ([#705]).
+- Support removing properties from catalogs.
+  This is helpful, because Trino fails to start in case you have any unused config properties ([#713]).
 - Support `access-control.properties` in configOverrides ([#721]).
 
 ### Changed
@@ -38,6 +40,7 @@ All notable changes to this project will be documented in this file.
 [#694]: https://github.com/stackabletech/trino-operator/pull/694
 [#695]: https://github.com/stackabletech/trino-operator/pull/695
 [#705]: https://github.com/stackabletech/trino-operator/pull/705
+[#713]: https://github.com/stackabletech/trino-operator/pull/713
 [#715]: https://github.com/stackabletech/trino-operator/pull/715
 [#717]: https://github.com/stackabletech/trino-operator/pull/717
 [#721]: https://github.com/stackabletech/trino-operator/pull/721

deploy/helm/trino-operator/crds/crds.yaml

@@ -2027,6 +2027,17 @@ spec:
                       description: A [TPC-H](https://docs.stackable.tech/home/nightly/trino/usage-guide/catalogs/tpch) connector.
                       type: object
                   type: object
+                experimentalConfigRemovals:
+                  default: []
+                  description: |-
+                    List of config properties which should be removed.
+
+                    This is helpful, because Trino fails to start in case you have any unused config properties. The removals are executed after the `configOverrides`.
+
+                    This field is experimental, and might be replaced by a more generic mechanism to edit config properties
+                  items:
+                    type: string
+                  type: array
               required:
                 - connector
               type: object

docs/modules/trino/pages/usage-guide/catalogs/index.adoc

@@ -31,9 +31,6 @@ spec:
           accessStyle: Path
           credentials:
             secretClass: minio-credentials
-# We can use configOverrides to add arbitrary properties to the Trino catalog configuration
-  configOverrides:
-    hive.metastore.username: trino
 ---
 apiVersion: trino.stackable.tech/v1alpha1
 kind: TrinoCatalog
@@ -60,6 +57,29 @@ The `metadata.labels` are used by TrinoCluster to determine the link between Tri
 The `spec.connector.<connector>` determines which connector is used.
 Each connector supports a different set of attributes.
 
+=== Config overrides and config removals
+
+You can use `.spec.configOverrides` to set arbitrary additional properties, which will be added to the catalog.
+
+There is also `.spec.experimentalConfigRemovals` to remove any properties the operator might set, but are not used by Trino.
+This causes Trino to refuse to startup with an error message such as `Error: Configuration property 'hive.s3.aws-access-key' was not used`.
+By removing the unneeded properties you can get Trino to start again.
+
+This example illustrates how to use config overrides and config removals
+
+[source,yaml]
+----
+apiVersion: trino.stackable.tech/v1alpha1
+kind: TrinoCatalog
+spec:
+  # Add some properties
+  configOverrides:
+    hive.metastore.username: trino
+  # Remove some properties
+  experimentalConfigRemovals:
+    - hive.s3.aws-access-key
+----
+
 === Add a catalog to a Trino cluster
 
 It is necessary to specify within the TrinoCluster which catalogs it should use.

rust/operator-binary/src/catalog/config.rs

@@ -161,6 +161,16 @@ impl CatalogConfig {
             .properties
             .extend(catalog.spec.config_overrides.clone());
 
+        for removal in &catalog.spec.config_removals {
+            if catalog_config.properties.remove(removal).is_none() {
+                tracing::warn!(
+                    catalog.name = catalog_name,
+                    property = removal,
+                    "You asked to remove a non-existing config property from a catalog"
+                );
+            }
+        }
+
         Ok(catalog_config)
     }
 }

rust/operator-binary/src/crd/catalog/mod.rs

@@ -49,10 +49,19 @@ pub mod versioned {
         /// The `connector` defines which connector is used.
         pub connector: TrinoCatalogConnector,
 
-        #[serde(default)]
         /// The `configOverrides` allow overriding arbitrary Trino settings.
         /// For example, for Hive you could add `hive.metastore.username: trino`.
+        #[serde(default)]
         pub config_overrides: HashMap<String, String>,
+
+        /// List of config properties which should be removed.
+        ///
+        /// This is helpful, because Trino fails to start in case you have any unused config
+        /// properties. The removals are executed after the `configOverrides`.
+        ///
+        /// This field is experimental, and might be replaced by a more generic mechanism to edit config properties
+        #[serde(default, rename = "experimentalConfigRemovals")]
+        pub config_removals: Vec<String>,
     }
 }