Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Resource permissions and sharing #166

Draft
wants to merge 113 commits into
base: main
Choose a base branch
from

Conversation

DarshitChanpura
Copy link
Owner

@DarshitChanpura DarshitChanpura commented Aug 30, 2024

@DarshitChanpura
Copy link
Owner Author

From latest ./gradlew run, it recognizes NoOpResourcePlugin.

[2024-08-30T12:42:17,976][INFO ][o.o.e.ExtensionsManager  ] [runTask-0] ExtensionsManager initialized
[2024-08-30T12:42:19,022][INFO ][o.o.a.r.ResourceService  ] [runTask-0] Security plugin disabled: Using NoOpResourcePlugin

Next test would be to determine whether it recognizes security plugin

@DarshitChanpura
Copy link
Owner Author

security plugin bootstrap: DarshitChanpura/security#10 (comment)

@DarshitChanpura DarshitChanpura changed the base branch from main to feature/segment-replication September 6, 2024 17:21
@DarshitChanpura DarshitChanpura changed the base branch from feature/segment-replication to main September 6, 2024 17:21
DarshitChanpura and others added 11 commits September 10, 2024 12:50
…arch-project#15893)

* Update TESTING.md 'Building with extra plugins' documentation

Signed-off-by: Finn Carroll <[email protected]>

* Typo

Signed-off-by: Finn Carroll <[email protected]>

* Fix TOC

Signed-off-by: Finn Carroll <[email protected]>

---------

Signed-off-by: Finn Carroll <[email protected]>
* fix case insensitive query on wildcard field

Signed-off-by: gesong.samuel <[email protected]>

* fix YAML test

Signed-off-by: gesong.samuel <[email protected]>

* add change log

Signed-off-by: gesong.samuel <[email protected]>

---------

Signed-off-by: gesong.samuel <[email protected]>
Co-authored-by: gesong.samuel <[email protected]>
)

Bumps com.gradle.develocity from 3.18 to 3.18.1.

---
updated-dependencies:
- dependency-name: com.gradle.develocity
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…s/hdfs-fixture (opensearch-project#15946)

* Bump ch.qos.logback:logback-core in /test/fixtures/hdfs-fixture

Bumps [ch.qos.logback:logback-core](https://github.com/qos-ch/logback) from 1.5.6 to 1.5.8.
- [Commits](qos-ch/logback@v_1.5.6...v_1.5.8)

---
updated-dependencies:
- dependency-name: ch.qos.logback:logback-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* Remote publication checksum stats

Signed-off-by: Himshikha Gupta <[email protected]>
Co-authored-by: Himshikha Gupta <[email protected]>
dependabot bot and others added 10 commits October 1, 2024 09:14
…s/repository-azure (opensearch-project#16133)

* Bump com.azure:azure-core-http-netty in /plugins/repository-azure

Bumps [com.azure:azure-core-http-netty](https://github.com/Azure/azure-sdk-for-java) from 1.15.3 to 1.15.4.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-core-http-netty_1.15.3...azure-core-http-netty_1.15.4)

---
updated-dependencies:
- dependency-name: com.azure:azure-core-http-netty
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…ture (opensearch-project#16135)

* Bump org.jline:jline in /test/fixtures/hdfs-fixture

Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.3 to 3.27.0.
- [Release notes](https://github.com/jline/jline3/releases)
- [Changelog](https://github.com/jline/jline3/blob/master/changelog.md)
- [Commits](jline/jline3@jline-parent-3.26.3...jline-3.27.0)

---
updated-dependencies:
- dependency-name: org.jline:jline
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* Remove Identity FeatureFlag

Signed-off-by: Craig Perkins <[email protected]>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <[email protected]>

* Mark IdentityService as internal

Signed-off-by: Craig Perkins <[email protected]>

* Add @internalapi annotation

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
…ed handler (opensearch-project#16154)

* Ensure RestHandler.Wrapper delegates all implementations to the wrapper handler

Signed-off-by: Craig Perkins <[email protected]>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <[email protected]>

* Fix typo

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
* Fix client-rest-high-level tests for ppc64le

Signed-off-by: prachi-gaonkar <[email protected]>

* Fix client-rest-high-level tests for ppc64le

Signed-off-by: prachi-gaonkar <[email protected]>

---------

Signed-off-by: prachi-gaonkar <[email protected]>
* Add bwc version 2.17.2

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Update Version.java

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Signed-off-by: Darshit Chanpura <[email protected]>
DarshitChanpura and others added 17 commits October 2, 2024 16:44
Signed-off-by: Darshit Chanpura <[email protected]>
Signed-off-by: Darshit Chanpura <[email protected]>
…on dynamic limit settings (opensearch-project#15986)

* Add changes to block calls in cat shards, indices and segments based on dynamic limit settings

Signed-off-by: Sumit Bansal <[email protected]>
…pensearch-project#16080)

* Separate Remote State and Publication enabled and configured methods

Signed-off-by: Shivansh Arora <[email protected]>
* add `Strings#isDigits` API

inspiration taken from [this SO answer][SO].

note that the stream is not parallelised to avoid the overhead of this
as the method is intended to be called primarily with shorter strings
where the time to set up would take longer than the actual check.

[SO]: https://stackoverflow.com/a/35150400

Signed-off-by: Ralph Ursprung <[email protected]>

* add `phone` & `phone-search` analyzer + tokenizer

this is largely based on [elasticsearch-phone] and internally uses
[libphonenumber].
this intentionally only ports a subset of the features: only `phone` and
`phone-search` are supported right now, `phone-email` can be added
if/when there's a clear need for it.

using `libphonenumber` is required since parsing phone numbers is a
non-trivial task (even though it might seem trivial at first glance!),
as can be seen in the list [falsehoods programmers believe about phone
numbers][falsehoods].

this allows defining the region to be used when analysing a phone
number. so far only the generic "unkown" region (`ZZ`) had been used
which worked as long as international numbers were prefixed with `+` but
did not work when using local numbers (e.g. a number stored as
`+4158...` was not matched against a number entered as `004158...` or
`058...`).

example configuration for an index:
```json
{
  "index": {
    "analysis": {
      "analyzer": {
        "phone": {
          "type": "phone"
        },
        "phone-search": {
          "type": "phone-search"
        },
        "phone-ch": {
          "type": "phone",
          "phone-region": "CH"
        },
        "phone-search-ch": {
          "type": "phone-search",
          "phone-region": "CH"
        }
      }
    }
  }
}
```
this creates four analyzers: `phone` and `phone-search` which do not
explicitly specify a region and thus fall back to `ZZ` (unknown region,
regional version of international dialing prefix (e.g. `00` instead of
`+` in most of europe) will not be recognised) and `phone-ch` and
`phone-search-ch` which will try to parse the phone number as a swiss
phone number (thus e.g. `00` as a prefix is recognised as the
international dialing prefix).

note that the analyzer is (currently) not meant to find phone numbers in
large text documents - instead it should be used on fields which contain
just the phone number (though extra text will be ignored) and it
collects the whole content of the field into a `String` in memory,
making it unsuitable for large field values.

this has been implemented in a new plugin which is however part of the
central opensearch repository as it was deemed too big an overhead to
have it in a separate repository but not important enough to bundle it
directly in `analysis-common` (see the discussion on the issue and the
PR for further details).

note that the new plugin has been added to the exclude list of the
javadoc check as this check is overzealous and also complains in many
cases where it shouldn't (e.g. on overridden methods - which it should
theoretically not do - or constructors which don't even exist). the
check first needs to be improved before this exclusion could be removed.

closes opensearch-project#11326

[elasticsearch-phone]: https://github.com/purecloudlabs/elasticsearch-phone
[libphonenumber]: https://github.com/google/libphonenumber
[falsehoods]: https://github.com/google/libphonenumber/blob/master/FALSEHOODS.md

Signed-off-by: Ralph Ursprung <[email protected]>

---------

Signed-off-by: Ralph Ursprung <[email protected]>
…pensearch-project#16194)

* Fix warnings from SLF4J on startup when repository-s3 is installed

Signed-off-by: Craig Perkins <[email protected]>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <[email protected]>

* Fix precommit

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Darshit Chanpura <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.