Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Resource permissions and sharing #166

Draft
wants to merge 113 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
909a85b
Adds a new plugin type named ResourcePlugin and relevant base classes
DarshitChanpura Aug 27, 2024
66a849c
Adds a No-op implementation of ResourcePlugin
DarshitChanpura Aug 27, 2024
08cdcb3
Merge remote-tracking branch 'upstream/main' into resource-permissions
DarshitChanpura Aug 30, 2024
d7169e4
Adds a way to configure security plugin for resource access-control
DarshitChanpura Aug 30, 2024
58ae851
Fixes compilation errors and changes debug log-level to info for Reso…
DarshitChanpura Aug 30, 2024
fd00243
Replace plugin count check with isEmpty
DarshitChanpura Aug 30, 2024
ef8a0b7
Adds package-info
DarshitChanpura Aug 30, 2024
e98cb61
Renames a bunch of files
DarshitChanpura Aug 30, 2024
96f09b0
Changes method signatures to be inline with their usage
DarshitChanpura Aug 30, 2024
c86dfc9
Adds new method for deleting by entity
DarshitChanpura Aug 30, 2024
7c6ec2a
Adds abstract method definitions for ResourcePlugin interface
DarshitChanpura Sep 3, 2024
c04762e
Merge remote-tracking branch 'upstream/main' into resource-permissions
DarshitChanpura Sep 3, 2024
f95a67f
Adds toXContent implementations
DarshitChanpura Sep 6, 2024
8b8fffd
Merge remote-tracking branch 'upstream/main' into resource-permissions
DarshitChanpura Sep 6, 2024
7e7cd0a
Modifies some method names and comments
DarshitChanpura Sep 10, 2024
4223fab
Removing task Id from map using remove() (#15918)
sgup432 Sep 12, 2024
07029b2
Add the capability to override indices.breaker.total.use_real_memory …
navneet1v Sep 13, 2024
260edc5
Fix flaky org.opensearch.rest.ReactorNetty4StreamingStressIT.testClos…
reta Sep 13, 2024
36c89bf
Update TESTING.md 'Building with extra plugins' documentation (#15893)
finnegancarroll Sep 13, 2024
aaa92ae
Fix case insensitive query on wildcard field (#15882)
HUSTERGS Sep 13, 2024
bd26056
Remove TooManyShardsInSnapshotsStatusException (#15877)
ltaragi Sep 16, 2024
c709400
Revert "Mute failing snapshot status tests (#15652)" (#15876)
ltaragi Sep 16, 2024
45a8ed4
Bump com.gradle.develocity from 3.18 to 3.18.1 (#15947)
dependabot[bot] Sep 16, 2024
e3bbc74
Bump ch.qos.logback:logback-core from 1.5.6 to 1.5.8 in /test/fixture…
dependabot[bot] Sep 16, 2024
81288b1
remote publication checksum stats (#15957)
himshikha Sep 17, 2024
8347d0e
Add validation for the search backpressure cancellation settings (#15…
gaobinlong Sep 17, 2024
eb5b703
Fix flaky terminaton conditions for org.opensearch.rest.ReactorNetty4…
reta Sep 17, 2024
7c427d9
Implement WithFieldName interface in ValuesSourceAggregationBuilder &…
dzane17 Sep 17, 2024
b2a7136
Bump com.microsoft.azure:msal4j from 1.17.0 to 1.17.1 in /plugins/rep…
dependabot[bot] Sep 18, 2024
4802d0d
Update version in README to 2.18.0 (#15968)
cwperks Sep 18, 2024
3937ccb
[AUTO] [main] Add bwc version 2.17.1. (#15966)
opensearch-trigger-bot[bot] Sep 18, 2024
3a1b6d1
Refactor branch deletion for backports (#15983)
gaiksaya Sep 18, 2024
80ff07e
star tree file formats refactoring and fixing offset bug (#15975)
bharath-techie Sep 19, 2024
6bef1e1
Fix infinite loop in nested agg (#15931)
kkewwei Sep 19, 2024
620db0a
Explicitly set include discovery nodes flag in reloadSecureSetting re…
SwethaGuptha Sep 19, 2024
ab7816c
Update version check in yaml test file for bitmap filtering (#15993)
gaobinlong Sep 19, 2024
77ddfd6
Fix deleting backport branches workflow (#16003)
gaiksaya Sep 19, 2024
1bc81d3
Remove identity-related feature flagged code from the RestController …
cwperks Sep 19, 2024
19d8df0
Update version check in yml test file (#15999)
reta Sep 19, 2024
94222f1
Update big5 workload config to use the ordered documents snapshot (#1…
rishabh6788 Sep 19, 2024
b369611
Add successfulSearchShardIndices in searchRequestContext before onReq…
dzane17 Sep 19, 2024
9537d39
add concurrency check to gradle-check workflow (#16007)
rishabh6788 Sep 20, 2024
2b670cc
Add support for async deletion in S3BlobContainer (#15621)
ashking94 Sep 20, 2024
f0ea056
Update protobuf from 3.25.4 to 3.25.5 (#16011)
cwperks Sep 20, 2024
4963792
Add more yaml rest tests for search backpressure settings (#16012)
gaobinlong Sep 20, 2024
05dab3b
Avoid infinite loop in flat_object parsing (#15985)
msfroh Sep 20, 2024
036f6bc
Add disallow settings update during repository in use ITs (#16001)
ashking94 Sep 21, 2024
1343367
Fixing flaky test testRemoteIndexPathFileExistsAfterMigration (#15977)
astute-decipher Sep 23, 2024
b984b9f
Bump actions/github-script from 5 to 7 (#16039)
dependabot[bot] Sep 23, 2024
c85ee68
Fix search_as_you_type not supporting multi-fields (#15988)
gaobinlong Sep 23, 2024
eeb2f39
Fix flaky test in RemoteStoreMigrationSettingsUpdateIT (#16048)
ashking94 Sep 24, 2024
d6bda7d
Bump dnsjava:dnsjava from 3.6.1 to 3.6.2 in /test/fixtures/hdfs-fixtu…
dependabot[bot] Sep 24, 2024
848e5c6
Update version check in yaml test file for the bug fix of avoid infin…
gaobinlong Sep 24, 2024
6a29119
Add support for docker compose v2 in TestFixturesPlugin (#16049)
cwperks Sep 24, 2024
b3cc802
Change the version to 2.18 as term-check fallback is merged to 2.x (#…
rajiv-kv Sep 24, 2024
12dadcf
Retry remote state download while bootstrap (#15950)
soosinha Sep 25, 2024
dc4dbce
Optimize remote store GC flow with pinned timestamps (#15943)
sachinpkale Sep 25, 2024
0617d95
Modify flaky test (#15854)
shiv0408 Sep 25, 2024
f1acc7a
Fix issue of red index on close for remote enabled clusters (#15990)
ashking94 Sep 25, 2024
a42e51d
Fix segment count for optimized agg rest test (#16067)
finnegancarroll Sep 25, 2024
daf1669
Added support for msearch API to pass search pipeline name (#15923)
owaiskazi19 Sep 26, 2024
ae22e3f
Update 390_search_as_you_type.yml (#16065)
reta Sep 26, 2024
5888867
Allow RemotePublication configured nodes to join remote cluster in mi…
rajiv-kv Sep 26, 2024
949b31f
Ignore google-cloud-storage and google-api-client major version upgra…
cwperks Sep 26, 2024
007d1b9
Bump com.nimbusds:nimbus-jose-jwt from 9.40 to 9.41.1 in /plugins/rep…
dependabot[bot] Sep 26, 2024
7caca26
Updates version to fix BWC for SearchSourceBuilder (#16086)
owaiskazi19 Sep 26, 2024
e8b02c9
Shallow snapshot v2 - create snapshot validations in a cluster state …
gbbafna Sep 27, 2024
8d19cb9
Bump org.roaringbitmap:RoaringBitmap from 1.2.1 to 1.3.0 in /server (…
dependabot[bot] Sep 27, 2024
0363a64
Bump com.nimbusds:nimbus-jose-jwt in /test/fixtures/hdfs-fixture (#16…
dependabot[bot] Sep 27, 2024
78860ac
Registering disable user-agent setting with OpenSearch (#16052)
saratvemulapalli Sep 27, 2024
b50117b
Bump com.maxmind.geoip2:geoip2 from 4.2.0 to 4.2.1 in /modules/ingest…
dependabot[bot] Sep 28, 2024
1563e1a
Fix for race condition in node-join/node-left loop (#15521)
rahulkarajgikar Sep 28, 2024
d510b12
Update Gradle to 8.10.2 (#16107)
reta Sep 29, 2024
031d8ae
Fix testHAProxyModeConnectionWorks is flaky (#16087)
gaobinlong Sep 29, 2024
0dba562
Implementing pagination for _cat/indices API (#14718)
gargharsh3134 Sep 30, 2024
1bddf2f
Bugfix in RemoteFsTimestampAwareTranslog.trimUnreferencedReaders (#16…
sachinpkale Sep 30, 2024
7ba8b78
Correct the version post backport to 2.18 instead of current (#16127)
gbbafna Sep 30, 2024
8ddb3ee
Add success and failure count OTel metrics for async shard fetch (#15…
rahulkarajgikar Sep 30, 2024
e6e290d
Fix version check in yml test file for double range profiler shows fi…
gaobinlong Sep 30, 2024
0b96565
Bump com.maxmind.db:maxmind-db from 3.1.0 to 3.1.1 in /modules/ingest…
dependabot[bot] Sep 30, 2024
9a5c7b2
Add default implementation to new finalizeSnapshot() in Repository (#…
sachinpkale Sep 30, 2024
d131d58
Bump org.apache.logging.log4j:log4j-core from 2.24.0 to 2.24.1 in /bu…
dependabot[bot] Sep 30, 2024
210228f
Add 2.17.1 release notes (#16104) (#16105) (#16132)
reta Oct 1, 2024
43e7597
Add assertBusy to failing snapshot tests (#16146)
ltaragi Oct 1, 2024
a767e92
Optimize checksum creation for remote cluster state (#16046)
himshikha Oct 1, 2024
1ef6444
Bump com.azure:azure-core-http-netty from 1.15.3 to 1.15.4 in /plugin…
dependabot[bot] Oct 1, 2024
be9f942
[SnapshotV2] Add timestamp of last successful fetch of pinned timesta…
ltaragi Oct 1, 2024
c46bf41
Bump org.jline:jline from 3.26.3 to 3.27.0 in /test/fixtures/hdfs-fix…
dependabot[bot] Oct 1, 2024
7b248ef
Remove Identity FeatureFlag (#16024)
cwperks Oct 1, 2024
908fefe
Change default retry mechanism of s3 client to Standard Mode (#15978)
Arpit-Bandejiya Oct 1, 2024
0b1650d
Ensure RestHandler.Wrapper delegates all implementations to the wrapp…
cwperks Oct 1, 2024
1ee858f
[BUG] Streaming bulk request hangs (#16158)
reta Oct 1, 2024
79a2ec1
Fix client-rest-high-level tests for ppc64le (#16089)
prachi-gaonkar Oct 2, 2024
4e3a6d0
[AUTO] [main] Add bwc version 2.17.2. (#16166)
opensearch-trigger-bot[bot] Oct 2, 2024
e1a1b62
Merge remote-tracking branch 'upstream/main' into resource-permissions
DarshitChanpura Oct 2, 2024
23fcfba
Fixes license
DarshitChanpura Oct 2, 2024
fba48ab
Adds changelog entry
DarshitChanpura Oct 2, 2024
9cb8d0e
Adds a notion of scope
DarshitChanpura Oct 2, 2024
e7ad37d
Change version checks from CURRENT to 2.18 (#16174)
ltaragi Oct 3, 2024
bf6566e
Add changes to block calls in cat shards, indices and segments based …
sumitasr Oct 3, 2024
5771e81
Separate Remote State and Publication enabled and configured methods …
shiv0408 Oct 3, 2024
6020c58
[Snapshot V2] Remove orphan timestamps post create snapshot completio…
gbbafna Oct 3, 2024
3d7184b
Bump Netty to 4.1.114.Final (#16182)
reta Oct 3, 2024
d1fd47c
Implement phone number analyzer (#15915)
rursprung Oct 3, 2024
f9e0c85
Fix japicmp configuration by treating abstract-to-default method chan…
reta Oct 4, 2024
aef7eca
Add more unit tests for RemoteStoreUtils and RemoteFsTimestampAwareTr…
sachinpkale Oct 4, 2024
ba8f1be
Run queued operations post v2 operations completion (#16179)
gbbafna Oct 4, 2024
b06ddb6
Fix warnings from SLF4J on startup when repository-s3 is installed (#…
cwperks Oct 4, 2024
421a1cc
Fix unknown parameter source_remote_translog_repository bug (#16192)
sachinpkale Oct 4, 2024
848234e
Modifies sharedwith to accomodate scope
DarshitChanpura Oct 4, 2024
eaf0c6e
Adds missing JavaDoc
DarshitChanpura Oct 4, 2024
6a6e6f7
Merge remote-tracking branch 'upstream/main' into resource-permissions
DarshitChanpura Oct 4, 2024
566913a
Adds NamedWriteable capability and removes un-needed method
DarshitChanpura Oct 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.accesscontrol.resources;

import org.opensearch.core.xcontent.ToXContentFragment;
import org.opensearch.core.xcontent.XContentBuilder;

import java.io.IOException;

/**
* This class contains information on the creator of a resource.
* Creator can either be a user or a backend_role.
*
* @opensearch.experimental
*/
public class CreatedBy implements ToXContentFragment {

private String user;

private String backendRole;

public CreatedBy(String user, String backendRole) {
this.user = user;
this.backendRole = backendRole;
}

public String getBackendRole() {
return backendRole;
}

public void setBackendRole(String backendRole) {
this.backendRole = backendRole;
}

public String getUser() {
return user;
}

public void setUser(String user) {
this.user = user;
}

@Override
public String toString() {
return "CreatedBy {" + "user='" + user + '\'' + ", backendRole='" + backendRole + '\'' + '}';
}

@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject().field("user", user).field("backend_role", backendRole).endObject();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.accesscontrol.resources;

/**
* This enum contains the type of entities a resource can be shared with.
*
* @opensearch.experimental
*/
public enum EntityType {

USERS,

ROLES,

BACKEND_ROLES,
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

package org.opensearch.accesscontrol.resources;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.opensearch.OpenSearchException;
import org.opensearch.plugins.NoOpResourceAccessControlPlugin;
import org.opensearch.plugins.ResourceAccessControlPlugin;
import org.opensearch.plugins.ResourcePlugin;

import java.util.List;
import java.util.stream.Collectors;

/**
* Resource access control for OpenSearch
*
* @opensearch.experimental
* */
public class ResourceService {
private static final Logger log = LogManager.getLogger(ResourceService.class);

private final ResourceAccessControlPlugin resourceACPlugin;
private final List<ResourcePlugin> resourcePlugins;

public ResourceService(final List<ResourceAccessControlPlugin> resourceACPlugins, List<ResourcePlugin> resourcePlugins) {
this.resourcePlugins = resourcePlugins;

if (resourceACPlugins.isEmpty()) {
log.info("Security plugin disabled: Using NoOpResourceAccessControlPlugin");
resourceACPlugin = new NoOpResourceAccessControlPlugin();
} else if (resourceACPlugins.size() == 1) {
log.info("Security plugin enabled: Using OpenSearchSecurityPlugin");
resourceACPlugin = resourceACPlugins.get(0);
} else {
throw new OpenSearchException(
"Multiple resource access control plugins are not supported, found: "
+ resourceACPlugins.stream().map(Object::getClass).map(Class::getName).collect(Collectors.joining(","))
);
}
}

/**
* Gets the current ResourcePlugin to perform authorization
*/
public ResourceAccessControlPlugin getResourceAccessControlPlugin() {
return resourceACPlugin;
}

/**
* List active plugins that define resources
*/
public List<ResourcePlugin> listResourcePlugins() {
return resourcePlugins;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.accesscontrol.resources;

import org.opensearch.core.xcontent.ToXContentFragment;
import org.opensearch.core.xcontent.XContentBuilder;

import java.io.IOException;
import java.util.Objects;

/**
* A document in .resource_sharing index.
* Holds information about the resource (obtained from defining plugin's meta-data),
* the index which defines the resources, the creator of the resource,
* and the information on whom this resource is shared with.
*
* @opensearch.experimental
*/
public class ResourceSharing implements ToXContentFragment {

private String sourceIdx;

private String resourceId;

private CreatedBy createdBy;

private ShareWith shareWith;

public ResourceSharing(String sourceIdx, String resourceId, CreatedBy createdBy, ShareWith shareWith) {
this.sourceIdx = sourceIdx;
this.resourceId = resourceId;
this.createdBy = createdBy;
this.shareWith = shareWith;
}

public String getSourceIdx() {
return sourceIdx;
}

public void setSourceIdx(String sourceIdx) {
this.sourceIdx = sourceIdx;
}

public String getResourceId() {
return resourceId;
}

public void setResourceId(String resourceId) {
this.resourceId = resourceId;
}

public CreatedBy getCreatedBy() {
return createdBy;
}

public void setCreatedBy(CreatedBy createdBy) {
this.createdBy = createdBy;
}

public ShareWith getShareWith() {
return shareWith;
}

public void setShareWith(ShareWith shareWith) {
this.shareWith = shareWith;
}

@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ResourceSharing resourceSharing = (ResourceSharing) o;
return Objects.equals(getSourceIdx(), resourceSharing.getSourceIdx())
&& Objects.equals(getResourceId(), resourceSharing.getResourceId())
&& Objects.equals(getCreatedBy(), resourceSharing.getCreatedBy())
&& Objects.equals(getShareWith(), resourceSharing.getShareWith());
}

@Override
public int hashCode() {
return Objects.hash(getSourceIdx(), getResourceId(), getCreatedBy(), getShareWith());
}

@Override
public String toString() {
return "Resource {"
+ "sourceIdx='"
+ sourceIdx
+ '\''
+ ", resourceId='"
+ resourceId
+ '\''
+ ", createdBy="
+ createdBy
+ ", sharedWith="
+ shareWith
+ '}';
}

@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject()
.field("source_idx", sourceIdx)
.field("resource_id", resourceId)
.field("created_by", createdBy)
.field("share_with", shareWith)
.endObject();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.accesscontrol.resources;

import org.opensearch.core.xcontent.ToXContentFragment;
import org.opensearch.core.xcontent.XContentBuilder;

import java.io.IOException;
import java.util.List;

/**
* This class contains information about whom a resource is shared with.
* It could be a user-name, a role or a backend_role.
*
* @opensearch.experimental
*/
public class ShareWith implements ToXContentFragment {

private List<String> users;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see pushback on introducing these concepts into the core. Is there a way to make this generic and let a plugin define an implementation?


private List<String> roles;

private List<String> backendRoles;

public ShareWith(List<String> users, List<String> roles, List<String> backendRoles) {
this.users = users;
this.roles = roles;
this.backendRoles = backendRoles;
}

public List<String> getUsers() {
return users;
}

public void setUsers(List<String> users) {
this.users = users;
}

public List<String> getRoles() {
return roles;
}

public void setRoles(List<String> roles) {
this.roles = roles;
}

public List<String> getBackendRoles() {
return backendRoles;
}

public void setBackendRoles(List<String> backendRoles) {
this.backendRoles = backendRoles;
}

@Override
public String toString() {
return "ShareWith {" + "users=" + users + ", roles=" + roles + ", backendRoles=" + backendRoles + '}';
}

@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject().field("users", users).field("roles", roles).field("backend_roles", backendRoles).endObject();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

/**
* Actions that OpenSearch can take either on the data stored on disk or on other nodes.
*/
/*
* Modifications Copyright OpenSearch Contributors. See
* GitHub history for details.
*/

/**
* This package defines all classes required for Resource Sharing and Access Control
*/
package org.opensearch.accesscontrol.resources;
Loading
Loading