Register new routes via SDK as named routes (#827)

* WIP on Handler naming and SSL Signed-off-by: Craig Perkins <[email protected]> * Add concept of extension shortname via settings Signed-off-by: Craig Perkins <[email protected]> * WIP on extension ssl Signed-off-by: Craig Perkins <[email protected]> * Get registry from runner Signed-off-by: Craig Perkins <[email protected]> * Read settings from extension config file Signed-off-by: Craig Perkins <[email protected]> * Update license headers Signed-off-by: Craig Perkins <[email protected]> * Run spotlessApply Signed-off-by: Craig Perkins <[email protected]> * Remove authz changes and only keep TLS Signed-off-by: Craig Perkins <[email protected]> * Remove authz changes and only keep TLS Signed-off-by: Craig Perkins <[email protected]> * Remove authz changes and only keep TLS Signed-off-by: Craig Perkins <[email protected]> * Remove authz changes and only keep TLS Signed-off-by: Craig Perkins <[email protected]> * Remove authz changes and only keep TLS Signed-off-by: Craig Perkins <[email protected]> * Update cert generation documents Signed-off-by: Craig Perkins <[email protected]> * Re-add authz changes for sample Hello world extension Signed-off-by: Craig Perkins <[email protected]> * Add ssl.transport.enabled in ExtensionsRunner Signed-off-by: Craig Perkins <[email protected]> * Name all HelloWorld extension routes Signed-off-by: Craig Perkins <[email protected]> * Run spotlessApply Signed-off-by: Craig Perkins <[email protected]> * Merge main into branch Signed-off-by: Craig Perkins <[email protected]> * Add instructions for running in SSL only mode Signed-off-by: Craig Perkins <[email protected]> * Add all SSL settings to extension settings Signed-off-by: Craig Perkins <[email protected]> * Update TestExtensionsRunner Signed-off-by: Craig Perkins <[email protected]> * Set default enforce_hostname_verification Signed-off-by: Craig Perkins <[email protected]> * Run spotlessApply Signed-off-by: Craig Perkins <[email protected]> * Respond to code review feedback Signed-off-by: Craig Perkins <[email protected]> * fix merge conflicts Signed-off-by: Craig Perkins <[email protected]> * Fix typos in debug messages Signed-off-by: Craig Perkins <[email protected]> * Add docstrings Signed-off-by: Craig Perkins <[email protected]> * Address code review feedback Signed-off-by: Craig Perkins <[email protected]> * Remove duplicate Signed-off-by: Craig Perkins <[email protected]> * Remove duplicate Signed-off-by: Craig Perkins <[email protected]> * Create ExtensionRouteHandlerFactory Signed-off-by: Craig Perkins <[email protected]> * Remove extension: from action naming Signed-off-by: Craig Perkins <[email protected]> * Add javadoc Signed-off-by: Craig Perkins <[email protected]> * Fix test compilation errors Signed-off-by: Craig Perkins <[email protected]> * Consolidate registerHandler Signed-off-by: Craig Perkins <[email protected]> * Fix missed registerHandler usage Signed-off-by: Craig Perkins <[email protected]> * Fix javadoc Signed-off-by: Craig Perkins <[email protected]> * Add method to check if class is initialized Signed-off-by: Craig Perkins <[email protected]> * Fix failing tests Signed-off-by: Craig Perkins <[email protected]> * Update helloworld-settings Signed-off-by: Craig Perkins <[email protected]> * Run spotlessApply Signed-off-by: Craig Perkins <[email protected]> * Run spotlessApply Signed-off-by: Craig Perkins <[email protected]> * Add shortExtensionName to BaseExtensionRouteHandler Signed-off-by: Craig Perkins <[email protected]> * Adds support for legacy action names while registering extension routes on extension start up Signed-off-by: Darshit Chanpura <[email protected]> * Modifies sample hello extension to conform to the new registration scheme Signed-off-by: Darshit Chanpura <[email protected]> * Renames route handlers Signed-off-by: Darshit Chanpura <[email protected]> * Adds certificate generation script Signed-off-by: Darshit Chanpura <[email protected]> * Fixes spotless errors Signed-off-by: Darshit Chanpura <[email protected]> * Fixes Javadoc Signed-off-by: Darshit Chanpura <[email protected]> * Cleans up route handlers to be more readable and adds an interface Signed-off-by: Darshit Chanpura <[email protected]> * Updates test to reflect changes in route handler signatures Signed-off-by: Darshit Chanpura <[email protected]> * Fixes slf4j gradle build issue Signed-off-by: Darshit Chanpura <[email protected]> * Fixes replaced named route tests Signed-off-by: Darshit Chanpura <[email protected]> * Changes the way named routes are serialized to conform to core and fixes tests Signed-off-by: Darshit Chanpura <[email protected]> * Removes mention of shortNames and uses extensionName as permission prefix Signed-off-by: Darshit Chanpura <[email protected]> * Used builder for named routes Signed-off-by: Darshit Chanpura <[email protected]> * Fixes broken changes Signed-off-by: Darshit Chanpura <[email protected]> * Updates dev guide to state to use credentials when registering extension while security is enabled Signed-off-by: Darshit Chanpura <[email protected]> * Replaces NamedRouteHandler and update logic to map route handlers for Rest request Signed-off-by: Darshit Chanpura <[email protected]> * Fixes typos Signed-off-by: Darshit Chanpura <[email protected]> * Removes references to ReplaceNamedRouteHandlers and DeprecatedNamedRouteHandlers Signed-off-by: Darshit Chanpura <[email protected]> * Addresses PR feedback Signed-off-by: Darshit Chanpura <[email protected]> * Refactors ReplacedRoute and deprecated route handlers to use RestResponse Signed-off-by: Darshit Chanpura <[email protected]> * Forces httpcore5 to 5.2.2 Signed-off-by: Darshit Chanpura <[email protected]> * Updates documentation Signed-off-by: Darshit Chanpura <[email protected]> * Addresses PR feedback Signed-off-by: Darshit Chanpura <[email protected]> * Fixes broken reference due to changes in core 52a5e3f6e0ca599e3193807134ea42660ecdd195 Signed-off-by: Darshit Chanpura <[email protected]> * Removes extra resolutionStrategy Signed-off-by: Darshit Chanpura <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Darshit Chanpura <[email protected]> Co-authored-by: Craig Perkins <[email protected]>
opensearch-project · Jul 10, 2023 · 09c22b0 · 09c22b0
1 parent f833e0c
commit 09c22b0
Show file tree

Hide file tree

Showing 17 changed files with 357 additions and 167 deletions.
diff --git a/CREATE_YOUR_FIRST_EXTENSION.md b/CREATE_YOUR_FIRST_EXTENSION.md
@@ -164,28 +164,28 @@ import org.opensearch.sdk.rest.BaseExtensionRestHandler;
 public class CrudAction extends BaseExtensionRestHandler {
 
     @Override
-    protected List<RouteHandler> routeHandlers() {
+    public List<NamedRoute> routes() {
         return List.of(
-            new RouteHandler(Method.PUT, "/sample", createHandler),
-            new RouteHandler(Method.GET, "/sample/{id}", readHandler),
-            new RouteHandler(Method.POST, "/sample/{id}", updateHandler),
-            new RouteHandler(Method.DELETE, "/sample/{id}", deleteHandler)
+            new NamedRoute.Builder().method(Method.PUT).path("/sample").uniqueName("extension1:sample/create").handler(createHandler).build(),
+            new NamedRoute.Builder().method(Method.GET).path("/sample/{id}").uniqueName("extension1:sample/get").handler(readHandler).build(),
+            new NamedRoute.Builder().method(Method.POST).path("/sample/{id}").uniqueName("extension1:sample/post").handler(updateHandler).build(),
+            new NamedRoute.Builder().method(Method.DELETE).path("/sample/{id}").uniqueName("extension1:sample/delete").handler(deleteHandler).build()
         );
     }
 
-    Function<RestRequest, ExtensionRestResponse> createHandler = (request) -> {
+    Function<RestRequest, RestResponse> createHandler = (request) -> {
         return new ExtensionRestResponse(request, RestStatus.OK, "To be implemented");
     };
 
-    Function<RestRequest, ExtensionRestResponse> readHandler = (request) -> {
+    Function<RestRequest, RestResponse> readHandler = (request) -> {
         return new ExtensionRestResponse(request, RestStatus.OK, "To be implemented");
     };
 
-    Function<RestRequest, ExtensionRestResponse> updateHandler = (request) -> {
+    Function<RestRequest, RestResponse> updateHandler = (request) -> {
         return new ExtensionRestResponse(request, RestStatus.OK, "To be implemented");
     };
 
-    Function<RestRequest, ExtensionRestResponse> deleteHandler = (request) -> {
+    Function<RestRequest, RestResponse> deleteHandler = (request) -> {
         return new ExtensionRestResponse(request, RestStatus.OK, "To be implemented");
     };
 }
@@ -248,7 +248,7 @@ return createJsonResponse(request, RestStatus.OK, "_id", response.id());
 Finally, you have the following code:
 
 ```java
-Function<RestRequest, ExtensionRestResponse> createHandler = (request) -> {
+Function<RestRequest, RestResponse> createHandler = (request) -> {
     IndexResponse response;
     try {
         BooleanResponse exists = client.indices().exists(new ExistsRequest.Builder().index("crudsample").build());

diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md
@@ -146,14 +146,14 @@ To **run OpenSearch from a compiled binary**, follow these steps:
 - Start OpenSearch using `./bin/opensearch`.
 - Send the below sample REST API to initialize an extension
 ```bash
-curl -XPOST "localhost:9200/_extensions/initialize" -H "Content-Type:application/json" --data '{ \
-"name":"hello-world", \
-"uniqueId":"hello-world", \
-"hostAddress":"127.0.0.1", \
-"port":"4532", \
-"version":"1.0", \
-"opensearchVersion":"3.0.0", \
-"minimumCompatibleVersion":"3.0.0", \
+curl -XPOST "localhost:9200/_extensions/initialize" -H "Content-Type:application/json" --data '{
+"name":"hello-world",
+"uniqueId":"hello-world",
+"hostAddress":"127.0.0.1",
+"port":"4532",
+"version":"1.0",
+"opensearchVersion":"3.0.0",
+"minimumCompatibleVersion":"3.0.0",
 "dependencies":[{"uniqueId":"test1","version":"2.0.0"},{"uniqueId":"test2","version":"3.0.0"}] \
 }'
 ```
@@ -162,18 +162,20 @@ To **run OpenSearch from Gradle**, follow these steps:
 - Run `./gradlew run` to start OpenSearch.
 - Send the below sample REST API to initialize an extension
 ```bash
-curl -XPOST "localhost:9200/_extensions/initialize" -H "Content-Type:application/json" --data '{ \
-"name":"hello-world", \
-"uniqueId":"hello-world", \
-"hostAddress":"127.0.0.1", \
-"port":"4532", \
-"version":"1.0", \
-"opensearchVersion":"3.0.0", \
-"minimumCompatibleVersion":"3.0.0", \
-"dependencies":[{"uniqueId":"test1","version":"2.0.0"},{"uniqueId":"test2","version":"3.0.0"}] \
+curl -XPOST "localhost:9200/_extensions/initialize" -H "Content-Type:application/json" --data '{
+"name":"hello-world",
+"uniqueId":"hello-world",
+"hostAddress":"127.0.0.1",
+"port":"4532",
+"version":"1.0",
+"opensearchVersion":"3.0.0",
+"minimumCompatibleVersion":"3.0.0",
+"dependencies":[{"uniqueId":"test1","version":"2.0.0"},{"uniqueId":"test2","version":"3.0.0"}]
 }'
 ```
 
+Note: If the Security plugin is initialized in OpenSearch, use admin credentials to send extension initialization request.
+
 In response to the REST `/initialize` request, `ExtensionsManager` discovers the extension listening on a predefined port and executes the TCP handshake protocol to establish a data transfer connection. Then OpenSearch sends a request to the OpenSearch SDK for Java and, upon acknowledgment, the extension responds with its name. This name is logged in the terminal where OpenSearch is running:
 
 ```bash

diff --git a/PLUGIN_MIGRATION.md b/PLUGIN_MIGRATION.md
@@ -67,14 +67,38 @@ XContentParser parser = XContentType.JSON
 Other potential initialization values are:
 ```java
 this.environmentSettings = extensionsRunner.getEnvironmentSettings();
-this.transportService = extensionsRunner.getExtensionTransportService();
+this.transportService = extensionsRunner.getSdkTransportService().getTransportService();
 this.restClient = anomalyDetectorExtension.getRestClient();
 this.sdkClusterService = new SDKClusterService(extensionsRunner);
 ```
 
 Many of these components are also available via Guice injection.
 
-Optionally, change the `routes()` to `routeHandlers()`. Change `prepareRequest()` to `handleRequest()`.
+### Replace `Route` with `NamedRoute`
+Change `routes()` to be NamedRoutes. Here is a sample of an existing route converted to a named route:
+Before:
+```
+public List<Route> routes() {
+    return ImmutableList.of(
+            new Route(GET, "/uri")
+        );
+}
+```
+With new scheme:
+```
+private Function<RestRequest, RestResponse> uriHandler = () -> {};
+public List<NamedRoute> routes() {
+    return ImmutableList.of(
+            new NamedRoute.Builder().method(GET).path("/uri").uniqueName("extension:uri").handler(uriHandler).build()
+        );
+}
+```
+
+You can optionally also add `actionNames()` to this route. These should correspond to any current actions defined as permissions in roles.
+`actionNames()` serve as a valuable tool for converting plugins into extensions while maintaining compatibility with pre-defined reserved roles.
+Ensure that these name-to-route mappings are easily accessible to the cluster admins to allow granting access to these APIs.
+
+Change `prepareRequest()` to `handleRequest()`.
 
 ### Replace `BytesRestResponse` with `ExtensionRestResponse`
 

diff --git a/build.gradle b/build.gradle
@@ -189,16 +189,18 @@ dependencies {
     testRuntimeOnly("org.junit.platform:junit-platform-launcher:${junitPlatform}")
 
     configurations.all {
-        resolutionStrategy.force("jakarta.json:jakarta.json-api:${jakartaVersion}")
-        resolutionStrategy.force("com.fasterxml.jackson.core:jackson-databind:${jacksonDatabindVersion}")
-        resolutionStrategy.force("com.fasterxml.jackson.core:jackson-core:${jacksonDatabindVersion}")
-        resolutionStrategy.force("com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:${jacksonDatabindVersion}")
-        resolutionStrategy.force("com.fasterxml.jackson.dataformat:jackson-dataformat-smile:${jacksonDatabindVersion}")
-        resolutionStrategy.force("com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:${jacksonDatabindVersion}")
-        resolutionStrategy.force("org.apache.logging.log4j:log4j-api:${log4jVersion}")
-        resolutionStrategy.force("org.apache.logging.log4j:log4j-core:${log4jVersion}")
-        resolutionStrategy.force("org.apache.logging.log4j:log4j-jul:${log4jVersion}")
-        resolutionStrategy.force("org.slf4j:slf4j-api:${slf4jVersion}")
+        resolutionStrategy {
+            force("jakarta.json:jakarta.json-api:${jakartaVersion}")
+            force("com.fasterxml.jackson.core:jackson-databind:${jacksonDatabindVersion}")
+            force("com.fasterxml.jackson.core:jackson-core:${jacksonDatabindVersion}")
+            force("com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:${jacksonDatabindVersion}")
+            force("com.fasterxml.jackson.dataformat:jackson-dataformat-smile:${jacksonDatabindVersion}")
+            force("com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:${jacksonDatabindVersion}")
+            force("org.apache.logging.log4j:log4j-api:${log4jVersion}")
+            force("org.apache.logging.log4j:log4j-core:${log4jVersion}")
+            force("org.apache.logging.log4j:log4j-jul:${log4jVersion}")
+            force("org.slf4j:slf4j-api:${slf4jVersion}")
+        }
     }
 }
 
@@ -322,7 +324,7 @@ task closeTestExtension (type: Exec) {
 tasks.named("integTest").configure { finalizedBy(closeTestExtension) }
 
 testClusters.integTest {
-    extension(new ExtensionsProperties("${testExtensionYml.name}", "${testExtensionYml.uniqueId}", "${testExtensionYml.hostAddress}", "${testExtensionYml.port}", "${testExtensionYml.version}", "${testExtensionYml.opensearchVersion}", "${testExtensionYml.minimumCompatibleVersion}"))
+    extension(true)
     testDistribution = "ARCHIVE"
     // Cluster shrink exception thrown if we try to set numberOfNodes to 1, so only apply if > 1
     if (_numNodes > 1) numberOfNodes = _numNodes

diff --git a/config/certs/cert-gen.sh b/config/certs/cert-gen.sh
@@ -0,0 +1,34 @@
+#! /bin/bash
+
+openssl genrsa -out root-ca-key.pem 2048
+openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=US/ST=NEW YORK/L=BROOKLYN/O=OPENSEARCH/OU=SECURITY/CN=ROOT" -out root-ca.pem -days 730
+
+openssl genrsa -out extension-01-key-temp.pem 2048
+openssl pkcs8 -inform PEM -outform PEM -in extension-01-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out extension-01-key.pem
+openssl req -new -key extension-01-key.pem -subj "/C=US/ST=NEW YORK/L=BROOKLYN/O=OPENSEARCH/OU=SECURITY/CN=extension-01" -out extension-01.csr
+echo 'subjectAltName=DNS:extension-01' | tee -a extension-01.ext
+echo 'subjectAltName=IP:172.20.0.11' | tee -a extension-01.ext
+openssl x509 -req -in extension-01.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out extension-01.pem -days 730 -extfile extension-01.ext
+
+rm extension-01-key-temp.pem
+rm extension-01.csr
+rm extension-01.ext
+rm root-ca.srl
+
+openssl genrsa -out admin-key-temp.pem 2048
+openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
+openssl req -new -key admin-key.pem -subj "/C=US/ST=NEW YORK/L=BROOKLYN/O=OPENSEARCH/OU=SECURITY/CN=A" -out admin.csr
+openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730
+openssl genrsa -out os-node-01-key-temp.pem 2048
+openssl pkcs8 -inform PEM -outform PEM -in os-node-01-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out os-node-01-key.pem
+openssl req -new -key os-node-01-key.pem -subj "/C=US/ST=NEW YORK/L=BROOKLYN/O=OPENSEARCH/OU=SECURITY/CN=os-node-01" -out os-node-01.csr
+echo 'subjectAltName=DNS:os-node-01' | tee -a os-node-01.ext
+echo 'subjectAltName=IP:172.20.0.11' | tee -a os-node-01.ext
+openssl x509 -req -in os-node-01.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out os-node-01.pem -days 730 -extfile os-node-01.ext
+
+rm admin-key-temp.pem
+rm admin.csr
+rm os-node-01-key-temp.pem
+rm os-node-01.csr
+rm os-node-01.ext
+rm root-ca.srl
diff --git a/src/main/java/org/opensearch/sdk/ExtensionSettings.java b/src/main/java/org/opensearch/sdk/ExtensionSettings.java
@@ -53,6 +53,7 @@ public class ExtensionSettings {
     private String hostPort;
     private String opensearchAddress;
     private String opensearchPort;
+    private Map<String, String> securitySettings;
 
     /**
      * A set of keys for security settings related to SSL transport, keystore and truststore files, and hostname verification.
@@ -83,8 +84,6 @@ public class ExtensionSettings {
         SSL_TRANSPORT_TRUSTSTORE_TYPE
     );
 
-    private Map<String, String> securitySettings;
-
     /**
      * Jackson requires a no-arg constructor.
      */

diff --git a/src/main/java/org/opensearch/sdk/ExtensionsRunner.java b/src/main/java/org/opensearch/sdk/ExtensionsRunner.java
@@ -36,6 +36,7 @@
 import org.opensearch.sdk.handlers.ExtensionsInitRequestHandler;
 import org.opensearch.sdk.handlers.ExtensionsRestRequestHandler;
 import org.opensearch.sdk.handlers.UpdateSettingsRequestHandler;
+import org.opensearch.sdk.rest.BaseExtensionRestHandler;
 import org.opensearch.sdk.rest.ExtensionRestHandler;
 import org.opensearch.sdk.rest.ExtensionRestPathRegistry;
 import org.opensearch.tasks.TaskManager;
@@ -233,6 +234,9 @@ protected ExtensionsRunner(Extension extension) throws IOException {
         if (extension instanceof ActionExtension) {
             // store REST handlers in the registry
             for (ExtensionRestHandler extensionRestHandler : ((ActionExtension) extension).getExtensionRestHandlers()) {
+                if (extensionRestHandler instanceof BaseExtensionRestHandler) {
+                    ((BaseExtensionRestHandler) extensionRestHandler).setExtensionName(extensionSettings.getExtensionName());
+                }
                 extensionRestPathRegistry.registerHandler(extensionRestHandler);
             }
         }