Skip to content

Commit

Permalink
update known issues and readme
Browse files Browse the repository at this point in the history
Signed-off-by: Mohammad Hasnain Mohsin Rajan <[email protected]>
  • Loading branch information
hasnain2808 committed Oct 22, 2023
1 parent 0c494ae commit 2796168
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 57 deletions.
15 changes: 1 addition & 14 deletions KNOWN_ISSUES.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,2 @@
# Known issues

## Cache deadlock

All elasticsearch versions between 5.5.3 (included) and 6.3.0 (excluded) are affected by a [bug](https://github.com/elastic/elasticsearch/pull/30461) in the internal cache that may lead to a serious deadlock that could in theory leak all the search threads.
A possible workaround is to configure the cache to avoid as much as possible its eviction mechanism by setting these options in your `elasticsearch.yml` file:

```
ltr.caches.expire_after_write: 0
ltr.caches.expire_after_read: 0
ltr.caches.max_mem: 100mb
```

elasticsearch versions 5.6.10, 6.3.0 and onwards should include the bugfix for this bug.

2.6.0: yamlresttests fail; set -Dbuild.snapshot=false to build successfully
59 changes: 16 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,16 @@ The OpenSearch Learning to Rank plugin uses machine learning to improve search r

To install, you'd run a command like this but replacing with the appropriate prebuilt version zip:

| OS | Command |
|-------|--------------------------------------------------------------------------------------------------------------------------------------------|
| 1.0.0 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.0.0/ltr-1.5.4-os1.0.0.zip` |
| 1.1.0 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.1.0/ltr-1.5.4-os1.1.0.zip` |
| 1.2.0 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.0/ltr-1.5.4-os1.2.0.zip` |
| 1.2.2 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.2/ltr-1.5.4-os1.2.2.zip` |
| 1.2.3 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.3/ltr-1.5.4-os1.2.3.zip` |
| 2.2.1 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/2.2.1/ltr-2.0.0-os2.2.1.zip` |
| 2.5.0 | `bin/opensearch-plugin install https://github.com/gsingers/opensearch-learning-to-rank-base/releases/download/release-v2.1.0/ltr-plugin-v2.1.0.zip` |
| OS | Command |
|--------|--------------------------------------------------------------------------------------------------------------------------------------------|
| 1.0.0 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.0.0/ltr-1.5.4-os1.0.0.zip` |
| 1.1.0 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.1.0/ltr-1.5.4-os1.1.0.zip` |
| 1.2.0 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.0/ltr-1.5.4-os1.2.0.zip` |
| 1.2.2 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.2/ltr-1.5.4-os1.2.2.zip` |
| 1.2.3 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/1.2.3/ltr-1.5.4-os1.2.3.zip` |
| 2.2.1 | `bin/opensearch-plugin install https://github.com/aparo/opensearch-learning-to-rank/releases/download/2.2.1/ltr-2.0.0-os2.2.1.zip` |
| 2.5.0 | `bin/opensearch-plugin install https://github.com/gsingers/opensearch-learning-to-rank-base/releases/download/release-v2.1.0/ltr-plugin-v2.1.0.zip` |
| 2.6.0 | `bin/opensearch-plugin install https://github.com/opensearch-project/opensearch-learning-to-rank-base/releases/download/release-v2.6.0/ltr-plugin-v2.6.0.zip` |


(It's expected you'll confirm some security exceptions, you can pass `-b` to `opensearch-plugin` to automatically install)
Expand All @@ -26,58 +27,30 @@ If you already are running OpenSearch, don't forget to restart!

# Releases

Releases can be found at https://github.com/gsingers/opensearch-learning-to-rank-base/releases.
Releases can be found at https://github.com/opensearch-project/opensearch-learning-to-rank-base/releases.

## Releasing/Packaging


Releases are done through Github Workflows (see `.github/workflows` in the root directory) on an as needed basis. If you do `./gradlew build` as per above under building,
it will build all the artifacts that are in the release.

## About alpha releases

These releases are alpha because some issues with the tests due to securemock that depends on ElasticSearch security stuff.
And there are 14 failing tests.

```
Tests with failures:
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionDoubleQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionMissingQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionIntegerQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionShortQueryParameter
- com.o19s.es.ltr.feature.store.StoredFeatureSetParserTests.testExpressionInvalidQueryParameter
- com.o19s.es.termstat.TermStatQueryBuilderTests.testMustRewrite
- com.o19s.es.termstat.TermStatQueryBuilderTests.testToQuery
- com.o19s.es.termstat.TermStatQueryBuilderTests.testCacheability
- com.o19s.es.ltr.feature.store.StoredFeatureParserTests.testExpressionOptimization
- com.o19s.es.termstat.TermStatQueryTests.testEmptyTerms
- com.o19s.es.termstat.TermStatQueryTests.testUniqueCount
- com.o19s.es.termstat.TermStatQueryTests.testBasicFormula
- com.o19s.es.termstat.TermStatQueryTests.testQuery
- com.o19s.es.termstat.TermStatQueryTests.testMatchCount
228 tests completed, 14 failed
```


# Development

To build, you need to disable the Java security manager
To build, you need to explicitly enable Java security and disable snapshot builds (until the YamlRestTests are fixed):

./gradlew -Dtests.security.manager=false clean build
./gradlew -Dopensearch.version={opensearch-version-to-build-on} -Djava.security.manager=allow -Dbuild.snapshot=false

# Upgrading the OpenSearch Versions

1. Edit `gradle.properties` to have the appropriate versions (it's often easiest to go download the latest tarball from OpenSearch and simply check the versions that ship) and to increment the version of this plugin
2. Build and test as above
3. Update this README with the version info in the table above
4. Upgrade the Docker file versions in the `docker` directory
1. Build and test as above
2. Update this README with the version info in the table above
3. Upgrade the Docker file versions in the `docker` directory
4. Test the docker image, per below.

## Development Notes



# Docker

A custom image of [OpenSearch](https://hub.docker.com/r/opensearchproject/opensearch) with the [OpenSearch Learning to Rank plugin](https://github.com/gsingers/opensearch-learning-to-rank-base) installed.
Expand Down

0 comments on commit 2796168

Please sign in to comment.