Skip to content

Commit

Permalink
Add cache plugin and tiered cache documentation (opensearch-project#7052
Browse files Browse the repository at this point in the history
)

* Add cache plugin and tiered cache documentation

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update index.md

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update concepts.md (opensearch-project#7049)

Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update alert-finding-api.md (opensearch-project#7036)

Signed-off-by: Heather Halter <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* [DOC] Add urldecode processor documentation (opensearch-project#5994)

* Add urldecode processor documentation

Signed-off-by: Melissa Vagi <[email protected]>

* Add text and examples

Signed-off-by: Melissa Vagi <[email protected]>

* Update urldecode.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/urldecode.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/urldecode.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/urldecode.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/urldecode.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/urldecode.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update urldecode.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

---------

Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* clarify "contexts" (opensearch-project#7063)

Signed-off-by: Stavros Macrakis <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Add documentation for indices.replication.max_bytes_per_sec (opensearch-project#7048)

* Add documentation for indices.replication.max_bytes_per_sec

Signed-off-by: Marc Handalian <[email protected]>

* Update _install-and-configure/configuring-opensearch/index-settings.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Marc Handalian <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update concurrent search docs with composite aggs updates (opensearch-project#7043)

Signed-off-by: Jay Deng <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* add query_by_tokens option in Neural Sparse Search (opensearch-project#7040)

* query tokens

Signed-off-by: zhichao-aws <[email protected]>

* fix typo

Signed-off-by: zhichao-aws <[email protected]>

* typo

Signed-off-by: zhichao-aws <[email protected]>

* fix

Signed-off-by: zhichao-aws <[email protected]>

* Update _query-dsl/specialized/neural-sparse.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>

* Update _query-dsl/specialized/neural-sparse.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>

* Update _query-dsl/specialized/neural-sparse.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>

* Update _query-dsl/specialized/neural-sparse.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>

* Update _query-dsl/specialized/neural-sparse.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update bundled JDK version (April 2024 Patch releases) (opensearch-project#7031)

Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Updating documentation for ignore_unavailable search request parameter (opensearch-project#6987)

* Updating documentation for ignore_unavailable search request parameter

Signed-off-by: Ankit Jain <[email protected]>

* Update _api-reference/search.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>

* Update _api-reference/document-apis/delete-by-query.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>

* Update _api-reference/document-apis/update-by-query.md

Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>

---------

Signed-off-by: Ankit Jain <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Add documentation for hardware-accelerated compression codecs. (opensearch-project#6841)

* Add documentation for hardware-accelerated compression codecs.

Signed-off-by: Mulugeta Mammo <[email protected]>

* Fix based on the review from reviewdog.

Signed-off-by: Mulugeta Mammo <[email protected]>

* Use the registered symbol for Intel and Xeon.

Signed-off-by: mulugetam <[email protected]>

* Minor modifications to the text.

Signed-off-by: mulugetam <[email protected]>

* Rephrasing.

Signed-off-by: mulugetam <[email protected]>

* Paraphrasing.

Signed-off-by: mulugetam <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Update _im-plugin/index-codecs.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Update _install-and-configure/configuring-opensearch/index-settings.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Mulugeta Mammo <[email protected]>
Signed-off-by: mulugetam <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Correct the release version for the upgrade API feature (opensearch-project#6955)

Signed-off-by: Peter Nied <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* [DOC] Add user agent processor documentation (opensearch-project#5995)

* Add user agent processor documentation

Signed-off-by: Melissa Vagi <[email protected]>

* Add user agent processor documentation

Signed-off-by: Melissa Vagi <[email protected]>

* Add examples

Signed-off-by: Melissa Vagi <[email protected]>

* Add user agent processor documentation

Signed-off-by: Melissa Vagi <[email protected]>

* Add user agent processor documentation

Signed-off-by: Melissa Vagi <[email protected]>

* Add examples

Signed-off-by: Melissa Vagi <[email protected]>

* Add examples and text

Signed-off-by: Melissa Vagi <[email protected]>

* Update user-agent.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update user-agent.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/user-agent.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/user-agent.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/user-agent.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/user-agent.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/user-agent.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _ingest-pipelines/processors/user-agent.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update user-agent.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

---------

Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Add understanding results page (opensearch-project#6984)

* Add refresh API.

Signed-off-by: Archer <[email protected]>

* Add Understanding Results section

Signed-off-by: Archer <[email protected]>

* Additional feedback.

Signed-off-by: Archer <[email protected]>

* Rework important metrics section.

Signed-off-by: Archer <[email protected]>

* Add summary report breakdown

Signed-off-by: Archer <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Delete _api-reference/index-apis/refresh.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Update understanding-results.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Archer <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Add cluster setting for filter rewrite optimization in aggregation (opensearch-project#7033)

* Add cluster setting for filter rewrite optimization in aggregation

Signed-off-by: bowenlan-amzn <[email protected]>

* Update search-settings.md

Signed-off-by: Melissa Vagi <[email protected]>

---------

Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* remove-has-childreren (opensearch-project#7072)

Signed-off-by: Heather Halter <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update csp configuration instructions for OSD (opensearch-project#7026)

* Update docs about configuring frame-ancestors

Signed-off-by: Tianle Huang <[email protected]>

* update fgac info

Signed-off-by: Tianle Huang <[email protected]>

* add fgac info

Signed-off-by: Tianle Huang <[email protected]>

* fix vale

Signed-off-by: Tianle Huang <[email protected]>

* fix readme

Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update csp-dynamic-configuration.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _dashboards/csp/csp-dynamic-configuration.md

Signed-off-by: Melissa Vagi <[email protected]>

---------

Signed-off-by: Tianle Huang <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* adding do_not_fail_on_forbidden section to docs opensearch-project#4896 (opensearch-project#6958)

* adding do_not_fail_on_forbidden section to docs opensearch-project#4896

Signed-off-by: AntonEliatra <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Stephen Crawford <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Stephen Crawford <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Stephen Crawford <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update permissions.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Signed-off-by: Melissa Vagi <[email protected]>

* Update permissions.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update permissions.md

Signed-off-by: Melissa Vagi <[email protected]>

Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _security/access-control/permissions.md

Signed-off-by: Melissa Vagi <[email protected]>

---------

Signed-off-by: AntonEliatra <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Co-authored-by: Stephen Crawford <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* feat: fix overlap rate param (opensearch-project#7045)

Signed-off-by: Ian Menendez <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* adding kibana_server role specification and explanation opensearch-project#4094 (opensearch-project#7066)

* adding kibana_user role specification and explanation opensearch-project#4094

Signed-off-by: AntonEliatra <[email protected]>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>

* Update multi-tenancy-config.md

Signed-off-by: AntonEliatra <[email protected]>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>

* Update multi-tenancy-config.md

Signed-off-by: AntonEliatra <[email protected]>

---------

Signed-off-by: AntonEliatra <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* expanding on TrustStore and KeyStore opensearch-project#4578 opensearch-project#4060 (opensearch-project#7015)

* expanding on TrustStore and KeyStore opensearch-project#4578 opensearch-project#4060

Signed-off-by: AntonEliatra <[email protected]>

* expanding on TrustStore and KeyStore opensearch-project#4578 opensearch-project#4060

Signed-off-by: AntonEliatra <[email protected]>

* Update generate-certificates.md

Signed-off-by: AntonEliatra <[email protected]>

* Update opensearch-keystore.md

Signed-off-by: AntonEliatra <[email protected]>

* Apply suggestions from code review

Co-authored-by: Stephen Crawford <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>

* Update security-admin.md

Signed-off-by: AntonEliatra <[email protected]>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>

---------

Signed-off-by: AntonEliatra <[email protected]>
Co-authored-by: Stephen Crawford <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Add documentation for primary rebalancing (opensearch-project#7059)

* Add documentation for primary rebalacing

Signed-off-by: Arpit Bandejiya <[email protected]>

* Update _install-and-configure/configuring-opensearch/cluster-settings.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Arpit Bandejiya <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update stats.md (opensearch-project#7087)

Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* navbar yml update.  (opensearch-project#7091)

* Omitting the named anchor from header Download link.

Signed-off-by: Nathan Boot <[email protected]>

* Updates navbar yml.

---------

Signed-off-by: Nathan Boot <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Missed a few includes that are referred to by the navbar. (opensearch-project#7093)

* Omitting the named anchor from header Download link.

Signed-off-by: Nathan Boot <[email protected]>

* Updates navbar yml.

* Navbar sass and html includes required for navbar.

---------

Signed-off-by: Nathan Boot <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Add missing OPENSEARCH_INITIAL_ADMIN_PASSWORD for both apt/deb and yum/rpm (opensearch-project#7079)

* Add missing OPENSEARCH_INITIAL_ADMIN_PASSWORD  for apt install on debian.md

This page's install command when installing via rep package is missing `OPENSEARCH_INITIAL_ADMIN_PASSWORD`

Signed-off-by: Drew Miranda <[email protected]>

* More update on deb and rpm installation steps on 2.12 security changes

Signed-off-by: Peter Zhu <[email protected]>

* Move 2.12 top

Signed-off-by: Peter Zhu <[email protected]>

* fix formatting

Signed-off-by: Peter Zhu <[email protected]>

* More tweaks

Signed-off-by: Peter Zhu <[email protected]>

* More tweaks 2

Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/debian.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/debian.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/debian.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/debian.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/debian.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/rpm.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/rpm.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/rpm.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/rpm.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/rpm.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/rpm.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/rpm.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

* Add more things

Signed-off-by: Peter Zhu <[email protected]>

* Add more things

Signed-off-by: Peter Zhu <[email protected]>

* Update _install-and-configure/install-opensearch/debian.md

Signed-off-by: Peter Zhu <[email protected]>

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>

---------

Signed-off-by: Drew Miranda <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>
Co-authored-by: Peter Zhu <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update tiered-cache.md

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Update _search-plugins/caching/tiered-cache.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sagar <[email protected]>
Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Heather Halter <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Stavros Macrakis <[email protected]>
Signed-off-by: Marc Handalian <[email protected]>
Signed-off-by: Jay Deng <[email protected]>
Signed-off-by: zhichao-aws <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
Signed-off-by: Mulugeta Mammo <[email protected]>
Signed-off-by: mulugetam <[email protected]>
Signed-off-by: Peter Nied <[email protected]>
Signed-off-by: Archer <[email protected]>
Signed-off-by: bowenlan-amzn <[email protected]>
Signed-off-by: Heather Halter <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>
Signed-off-by: Tianle Huang <[email protected]>
Signed-off-by: AntonEliatra <[email protected]>
Signed-off-by: Ian Menendez <[email protected]>
Signed-off-by: Arpit Bandejiya <[email protected]>
Signed-off-by: Nathan Boot <[email protected]>
Signed-off-by: Drew Miranda <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>
Signed-off-by: Sagar <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Co-authored-by: Stavros Macrakis <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Jay Deng <[email protected]>
Co-authored-by: zhichao-aws <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: Ankit Jain <[email protected]>
Co-authored-by: mulugetam <[email protected]>
Co-authored-by: Peter Nied <[email protected]>
Co-authored-by: bowenlan-amzn <[email protected]>
Co-authored-by: Tianle Huang <[email protected]>
Co-authored-by: AntonEliatra <[email protected]>
Co-authored-by: Stephen Crawford <[email protected]>
Co-authored-by: Ian Menendez <[email protected]>
Co-authored-by: Arpit-Bandejiya <[email protected]>
Co-authored-by: Nate B <[email protected]>
Co-authored-by: Drew Miranda <[email protected]>
Co-authored-by: Peter Zhu <[email protected]>
  • Loading branch information
1 parent e630544 commit 6397916
Show file tree
Hide file tree
Showing 2 changed files with 129 additions and 0 deletions.
37 changes: 37 additions & 0 deletions _search-plugins/caching/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
layout: default
title: Caching
parent: Improving search performance
has_children: true
nav_order: 100
---

# Caching

OpenSearch relies on different on-heap cache types to accelerate data retrieval, providing significant improvement in search latency. However, cache size is limited by the amount of memory available on a node. When processing a larger dataset that can potentially be cached, the cache size limit can result in many pieces of data either being removed from the cache or not being cached, causing an incomplete query. This impacts performance because OpenSearch needs to process the query again, causing high resource consumption.

Understanding how your data uses the cache can help improve your cluster's performance and prevent you from using too much memory, reducing the cost of querying your data.

## Supported on-heap cache types

OpenSearch supports the following on-heap cache types:

- **Request cache**: Caches the local results on each shard. This allows frequently used and potentially resource-heavy search requests to return results almost instantaneously.
- **Query cache**: Caches common data from similar queries at the shard level. The query cache is more granular than the request cache and can cache data to be reused in different queries.
- **Field data cache**: Caches field data and global ordinals, which are both used to support aggregations on certain field types.

## Additional cache stores

**Introduced 2.14**
{: .label .label-purple }

This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/OpenSearch/issues/10024).
{: .warning}

In addition to existing custom OpenSearch on-heap cache stores, cache plugins provide the following cache stores:

- **Disk cache**: Stores the precomputed result of a query on disk. Use a disk cache to cache much larger datasets, provided that the disk's latency is within an acceptable range.
- **Tiered cache**: A multi-level cache in which each tier has its own characteristics and performance levels. For example, a tiered cache can contain both on-heap and disk tiers. By combining different tiers, you can achieve a balance between cache performance and size. To learn more, see [Tiered cache]({{site.url}}{{site.baseurl}}/search-plugins/caching/tiered-cache/).

In OpenSearch 2.14, the request cache is integrated with cache plugins. You can use a tiered or disk cache as a request-level cache.
{: .note}
92 changes: 92 additions & 0 deletions _search-plugins/caching/tiered-cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
layout: default
title: Tiered cache
parent: Caching
grand_parent: Improving search performance
nav_order: 10
---

# Tiered cache

This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/OpenSearch/issues/10024).
{: .warning}

A tiered cache is a multi-level cache in which each tier has its own characteristics and performance levels. By combining different tiers, you can achieve a balance between cache performance and size.

## Types of tiered caches

OpenSearch provides an implementation of a `_tiered` spillover `cache_`. This implementation spills any items removed from the upper tiers to the lower tiers of cache. The upper tier, such as the on-heap tier, is smaller in size but offers better latency. The lower tier, such as the disk cache, is larger in size but slower in terms of latency. OpenSearch offers both on-heap and disk tiers.

## Enabling a tiered cache

To enable a tiered cache, configure the following setting in `opensearch.yml`:

```yaml
opensearch.experimental.feature.pluggable.caching.enabled: true
```
{% include copy.html %}
For more information about ways to enable experimental features, see [Experimental feature flags]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/experimental/).
## Installing required plugins
To use tiered caching, install a tiered cache plugin. As of OpenSearch 2.13, the only available cache plugin is the `cache-ehcache` plugin. This plugin provides a disk cache implementation that can be used as a disk tier within a tiered cache. For more information about installing non-bundled plugins, see [Additional plugins]({{site.url}}{{site.baseurl}}/install-and-configure/plugins/#additional-plugins).

A tiered cache will fail to initialize if the `cache-ehcache` plugin is not installed or if disk cache properties are not set.
{: .warning}

## Tiered cache settings

In OpenSearch 2.14, a request cache can be used in a tiered cache. To begin, configure the following settings in the `opensearch.yml` file.

### Cache store name

To use the OpenSearch-provided tiered spillover cache implementation, set the cache store name to `tiered_spillover`, as shown in the following example:

```yaml
indices.request.cache.store.name: tiered_spillover: true
```
{% include copy.html %}

### Setting on-heap and disk store tiers

Set the on-heap and disk store tiers to `opensearch_onheap` and `ehcache_disk`, as shown in the following example:

```yaml
indices.request.cache.tiered_spillover.onheap.store.name: opensearch_onheap
indices.request.cache.tiered_spillover.disk.store.name: ehcache_disk
```
The `opensearch_onheap` setting uses the built-in on-heap cache available in OpenSearch.

The `ehcache_disk` setting is the disk cache implementation from [Ehcache](https://www.ehcache.org/) and requires installing the `cache-ehcache` plugin.

{% include copy.html %}

### Configuring on-heap and disk stores

The following table lists the cache store settings for the `opensearch_onheap` store.

Setting | Data type | Default | Description
:--- | :--- | :--- | :---
`indices.request.cache.opensearch_onheap.size` | Percentage | 1% of the heap size | The size of the on-heap cache. Optional.
`indices.request.cache.opensearch_onheap.expire` | Time unit | `MAX_VALUE` (disabled) | Specifies a time-to-live (TTL) for the cached results. Optional.

The following table lists the disk cache store settings for the `ehcache_disk` store.

Setting | Data type | Default | Description
:--- | :--- | :--- | :---
`indices.request.cache.ehcache_disk.max_size_in_bytes` | Long | `1073741824` (1 GB) | Defines the size of the disk cache. Optional.
`indices.request.cache.ehcache_disk.storage.path` | String | `""` | Defines the storage path for the disk cache. Required.
`indices.request.cache.ehcache_disk.expire_after_access` | Time unit | `MAX_VALUE` (disabled) | Specifies a TTL for the cached results. Optional.
`indices.request.cache.ehcache_disk.alias` | String | `ehcacheDiskCache#INDICES_REQUEST_CACHE` | Specifies an alias for the disk cache. Optional.
`indices.request.cache.ehcache_disk.segments` | Integer | `16` | Defines the number of segments into which the disk cache is separated. Used for concurrency. Optional.
`indices.request.cache.ehcache_disk.concurrency` | Integer | `1` | Defines the number of distinct write queues created for the disk store, where a group of segments shares a write queue. Optional.

### Additional settings for the `tiered_spillover` store

The following table lists additional settings for the `tiered_spillover` store setting.

Setting | Data type | Default | Description
:--- | :--- | :--- | :---
`indices.request.cache.tiered_spillover.disk.store.policies.took_time.threshold` | Time unit | `10ms` | A policy used to determine whether to cache a query into a disk cache based on its took time. This is a dynamic setting. Optional.
`indices.request.cache.tiered_spillover.disk.store.enabled` | Boolean | `True` | Enables or disables the disk cache dynamically within a tiered spillover cache. Note: After disabling a disk cache, entries are not removed automatically and requires the cache to be manually cleared. Optional.

0 comments on commit 6397916

Please sign in to comment.