[Rule-based Autotagging] add security attributes to rules #19232

ruai0511 · 2025-09-03T21:55:08Z

Description

This PR enhances rule-based auto-tagging functionality by introducing security-related attributes. Specifically, we aim to extract user information (username and role) from the security context of incoming requests. In many real-world environments, security context is essential for customers with multiple teams, departments, or security boundaries. By enabling the use of security attributes in auto-tagging rules, this project will provide more practical and convenient ways to define the tenants.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-09-03T22:22:17Z

❌ Gradle check result for 07a5cd4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Ruirui Zhang <[email protected]>

github-actions · 2025-09-04T00:04:54Z

❌ Gradle check result for eb67fce: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

peternied · 2025-09-04T17:47:48Z

@ruai0511 Thanks for this change this would be really useful for tracing requests to have these attributes. How will this change work with the subject from IdentityPlugin?

@cwperks love to get your thoughts on this change.

cwperks · 2025-09-04T17:49:01Z

modules/autotagging-commons/common/src/main/java/org/opensearch/rule/SecurityAttribute.java

+    /**
+     * Key representing the username subfield.
+     */
+    public static final String USERNAME = "username";


Can we make this more generic so that there's no reference to username or role?

I'm not sure how WLM works, but I know that a request will match to the best workload group. How does the scoring work for matching? Can we assign weights to different attributes to help with the matching problem to determine the best group?

cwperks · 2025-09-04T17:55:54Z

@cwperks love to get your thoughts on this change.

@peternied we've been discussion an extensible <-> extending plugin relationship here, especially since you can now define optional relationships with extending -> extensible plugin.

In this case, WLM needs attributes from security, but it differs from other plugins because its in the core. There's certainly prevalent usage of common-utils today but we should not introduce a dependency on common-utils in the core repo. (In general, we need to work towards removal of common-utils).

IdentityPlugin is one avenue, but that extension point is quite simple and only used for username and has no notion of roles or other attributes. We could certainly entertain expanding on the notion of subject to add roles, but I know there were differing opinions on that and as a result the IdentityPlugin can only be used to get username.

In this case, by making WLM extensible it can ask other plugins to define an attribute extractor to feed it back attributes (and weights to break ties) given a REST Request. For security that could be username + roles and it would give higher precedence to direct username matching to workload groups.

kaushalmahi12

I have the following initial set of comments which might change rest of the PR changes hence not reviewing it further.

kaushalmahi12 · 2025-09-04T18:29:14Z

...commons/common/src/main/java/org/opensearch/rule/attribute_extractor/AttributeExtractor.java

+     * Defines the combination style used when a request contains multiple values
+     * for an attribute.
+     */
+    enum CombinationStyle {


Lets rename this to LogicalOperator.

kaushalmahi12 · 2025-09-04T18:40:12Z

modules/autotagging-commons/common/src/main/java/org/opensearch/rule/autotagging/Attribute.java

+    /**
+     * Returns the allowed subfields ordered from highest to lowest priority
+     */
+    default List<String> getPrioritizedSubfields() {


If we are meaning to introduce hierarchical attributes then returning List makes sense. Since it can support the current use case and still keep the code closed for modification if we want to move to hierarchical model.

kaushalmahi12 · 2025-09-04T18:45:06Z

modules/autotagging-commons/common/src/main/java/org/opensearch/rule/autotagging/Attribute.java

+     * Parses attribute values for specific attributes
+     * @param parser the XContent parser
+     */
+    default Set<String> fromXContentParseAttributeValues(XContentParser parser) throws IOException {


I don't see it resonating with subfield/sub attribute model. Can we add some example usage in the comments to better comprehend the new Attribute design model

kaushalmahi12 · 2025-09-04T18:49:25Z

...es/autotagging-commons/common/src/main/java/org/opensearch/rule/autotagging/FeatureType.java

+     * Returns the list of attributes ordered by their processing priority.
+     * @return List of prioritized attributes.
+     */
+    List<Attribute> getPrioritizedAttributesList();


I think with this chances of errors are high since any modification or merge conflicts can easily mess up the supposed order. I would advise to keep the prioritization as an integer value within attribute definition and clients of the list can sort the attributes based on the priority of the attributes.

The proposed approach also can remove the need for this additional method.

But if we keep them in attribute definition, the ordering would be the same for all feature types. But ideally for each use case the order might be different.

kaushalmahi12 · 2025-09-04T18:53:35Z

...ing-commons/common/src/main/java/org/opensearch/rule/storage/DefaultAttributeValueStore.java

-            for (Map.Entry<String, V> possibleMatch : trie.prefixMap(longestMatchingPrefix).entrySet()) {
-                if (key.startsWith(possibleMatch.getKey())) {
-                    return Optional.of(possibleMatch.getValue());
+            for (int len = key.length(); len >= 0; len--) {


This is tricky and the time complexity here is changing to O(n^2). Can we come up with a more optimal implementation ?

modules/autotagging-commons/common/src/main/java/org/opensearch/rule/SecurityAttribute.java

github-actions · 2025-09-05T19:25:29Z

❌ Gradle check result for 756409d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Ruirui Zhang <[email protected]>

github-actions · 2025-09-05T22:11:44Z

❌ Gradle check result for 209f55d: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ruai0511 requested a review from a team as a code owner September 3, 2025 21:55

ruai0511 force-pushed the security-attr branch 2 times, most recently from 7d53efb to 07a5cd4 Compare September 3, 2025 22:05

ruai0511 added the v3.3.0 label Sep 3, 2025

add security attributes to auto tagging

eb67fce

Signed-off-by: Ruirui Zhang <[email protected]>

ruai0511 force-pushed the security-attr branch from 07a5cd4 to eb67fce Compare September 3, 2025 23:44

cwperks reviewed Sep 4, 2025

View reviewed changes

kaushalmahi12 reviewed Sep 4, 2025

View reviewed changes

ruai0511 mentioned this pull request Sep 5, 2025

Add logic to extract security attributes for rule-based autotagging opensearch-project/security#5606

Open

5 tasks

revise based on comments

209f55d

Signed-off-by: Ruirui Zhang <[email protected]>

ruai0511 force-pushed the security-attr branch from 756409d to 209f55d Compare September 5, 2025 20:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Rule-based Autotagging] add security attributes to rules #19232

[Rule-based Autotagging] add security attributes to rules #19232

ruai0511 commented Sep 3, 2025

Uh oh!

github-actions bot commented Sep 3, 2025

Uh oh!

github-actions bot commented Sep 4, 2025

Uh oh!

peternied commented Sep 4, 2025

Uh oh!

cwperks Sep 4, 2025

Uh oh!

cwperks commented Sep 4, 2025

Uh oh!

kaushalmahi12 left a comment

Uh oh!

kaushalmahi12 Sep 4, 2025

Uh oh!

kaushalmahi12 Sep 4, 2025

Uh oh!

kaushalmahi12 Sep 4, 2025

Uh oh!

kaushalmahi12 Sep 4, 2025

Uh oh!

ruai0511 Sep 4, 2025

Uh oh!

kaushalmahi12 Sep 4, 2025

Uh oh!

Uh oh!

github-actions bot commented Sep 5, 2025

Uh oh!

github-actions bot commented Sep 5, 2025

Uh oh!

Uh oh!

[Rule-based Autotagging] add security attributes to rules #19232

Are you sure you want to change the base?

[Rule-based Autotagging] add security attributes to rules #19232

Conversation

ruai0511 commented Sep 3, 2025

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Sep 3, 2025

Uh oh!

github-actions bot commented Sep 4, 2025

Uh oh!

peternied commented Sep 4, 2025

Uh oh!

cwperks Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

cwperks commented Sep 4, 2025

Uh oh!

kaushalmahi12 left a comment

Choose a reason for hiding this comment

Uh oh!

kaushalmahi12 Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

kaushalmahi12 Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

kaushalmahi12 Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

kaushalmahi12 Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

ruai0511 Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

kaushalmahi12 Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Sep 5, 2025

Uh oh!

github-actions bot commented Sep 5, 2025

Uh oh!

Uh oh!