Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main' into initial-changes-add…
Browse files Browse the repository at this point in the history
…-backport-automerge-label
  • Loading branch information
peterzhuamazon committed Apr 2, 2024
2 parents 3f65e40 + 9ff5db3 commit 3bb39c3
Show file tree
Hide file tree
Showing 13 changed files with 33 additions and 23 deletions.
1 change: 1 addition & 0 deletions _about/version-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ permalink: /version-history/

OpenSearch version | Release highlights | Release date
:--- | :--- | :---
[2.13.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.13.0.md) | Makes agents and tools and the OpenSearch Assistant Toolkit generally available. Introduces vector quantization within OpenSearch. Adds LLM guardrails and hybrid search with aggregations. Adds the Bloom filter skipping index for Apache Spark data sources, I/O-based admission control, and the ability to add an alerting cluster that manages all alerting tasks. For a full list of release highlights, see the Release Notes. | 2 April 2024
[2.12.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.12.0.md) | Makes concurrent segment search and conversational search generally available. Provides an experimental OpenSearch Assistant Toolkit, including agents and tools, workflow automation, and OpenSearch Assistant for OpenSearch Dashboards UI. Adds a new match-only text field, query insights to monitor top N queries, and k-NN search on nested fields. For a full list of release highlights, see the Release Notes. | 20 February 2024
[2.11.1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.11.1.md) | Includes maintenance changes and bug fixes for cross-cluster replication, alerting, observability, OpenSearch Dashboards, index management, machine learning, security, and security analytics. For a full list of release highlights, see the Release Notes. | 30 November 2023
[2.11.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.11.0.md) | Adds multimodal and sparse neural search capability and the ability to take shallow snapshots that refer to data stored in remote-backed storage. Makes the search comparison tool generally available. Includes a simplified workflow to create threat detectors in Security Analytics and improved security in OpenSearch Dashboards. Experimental features include a new framework and toolset for distributed tracing and updates to conversational search. For a full list of release highlights, see the Release Notes. | 16 October 2023
Expand Down
1 change: 1 addition & 0 deletions _data-prepper/pipelines/configuration/buffers/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ Option | Required | Type | Description
#### producer_properties

Use the following configuration options to configure a Kafka producer.

Option | Required | Type | Description
:--- | :--- | :--- | :---
`max_request_size` | No | Integer | The maximum size of the request that the producer sends to Kafka. Default is 1 MB.
Expand Down
6 changes: 5 additions & 1 deletion _query-dsl/term/exists.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,4 +146,8 @@ The response contains the matching document:

## Parameters

The query accepts the name of the field (`<field>`) as a top-level parameter.
The query accepts the name of the field (`<field>`) as a top-level parameter.

Parameter | Data type | Description
:--- | :--- | :---
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
5 changes: 3 additions & 2 deletions _query-dsl/term/fuzzy.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ GET _search
"fuzzy": {
"<field>": {
"value": "sample",
...
...
}
}
}
Expand All @@ -80,11 +80,12 @@ The `<field>` accepts the following parameters. All parameters except `value` ar
Parameter | Data type | Description
:--- | :--- | :---
`value` | String | The term to search for in the field specified in `<field>`.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
`fuzziness` | `AUTO`, `0`, or a positive integer | The number of character edits (insert, delete, substitute) needed to change one word to another when determining whether a term matched a value. For example, the distance between `wined` and `wind` is 1. The default, `AUTO`, chooses a value based on the length of each term and is a good choice for most use cases.
`max_expansions` | Positive integer | The maximum number of terms to which the query can expand. Fuzzy queries “expand to” a number of matching terms that are within the distance specified in `fuzziness`. Then OpenSearch tries to match those terms. Default is `50`.
`prefix_length` | Non-negative integer | The number of leading characters that are not considered in fuzziness. Default is `0`.
`rewrite` | String | Determines how OpenSearch rewrites and scores multi-term queries. Valid values are `constant_score`, `scoring_boolean`, `constant_score_boolean`, `top_terms_N`, `top_terms_boost_N`, and `top_terms_blended_freqs_N`. Default is `constant_score`.
`transpositions` | Boolean | Specifies whether to allow transpositions of two adjacent characters (`ab` to `ba`) as edits. Default is `true`.
`transpositions` | Boolean | Specifies whether to allow transpositions of two adjacent characters (`ab` to `ba`) as edits. Default is `true`.

Specifying a large value in `max_expansions` can lead to poor performance, especially if `prefix_length` is set to `0`, because of the large number of variations of the word that OpenSearch tries to match.
{: .warning}
Expand Down
1 change: 1 addition & 0 deletions _query-dsl/term/ids.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,4 @@ The query accepts the following parameter.
Parameter | Data type | Description
:--- | :--- | :---
`values` | Array of strings | The document IDs to search for. Required.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
5 changes: 3 additions & 2 deletions _query-dsl/term/prefix.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ GET _search
"prefix": {
"<field>": {
"value": "sample",
...
...
}
}
}
Expand All @@ -63,8 +63,9 @@ The `<field>` accepts the following parameters. All parameters except `value` ar
Parameter | Data type | Description
:--- | :--- | :---
`value` | String | The term to search for in the field specified in `<field>`.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
`case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping).
`rewrite` | String | Determines how OpenSearch rewrites and scores multi-term queries. Valid values are `constant_score`, `scoring_boolean`, `constant_score_boolean`, `top_terms_N`, `top_terms_boost_N`, and `top_terms_blended_freqs_N`. Default is `constant_score`.

If [`search.allow_expensive_queries`]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries) is set to `false`, prefix queries are not run. If `index_prefixes` is enabled, the `search.allow_expensive_queries` setting is ignored and an optimized query is built and run.
{: .important}
{: .important}
14 changes: 7 additions & 7 deletions _query-dsl/term/range.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ OpenSearch populates missing date components with the following values:
- `SECOND_OF_MINUTE`: `59`
- `NANO_OF_SECOND`: `999_999_999`

If the year is missing, it is not populated.
If the year is missing, it is not populated.

For example, consider the following request that specifies only the year in the start date:

Expand Down Expand Up @@ -131,7 +131,7 @@ GET products/_search
```
{% include copy-curl.html %}

In the preceding example, `2019/01/01` is the anchor date (the starting point) for the date math. After the two pipe characters (`||`), you are specifying a mathematical expression relative to the anchor date. In this example, you are subtracting 1 year (`-1y`) and 1 day (`-1d`).
In the preceding example, `2019/01/01` is the anchor date (the starting point) for the date math. After the two pipe characters (`||`), you are specifying a mathematical expression relative to the anchor date. In this example, you are subtracting 1 year (`-1y`) and 1 day (`-1d`).

You can also round off dates by adding a forward slash to the date or time unit.

Expand Down Expand Up @@ -175,16 +175,16 @@ GET /products/_search
"query": {
"range": {
"created": {
"time_zone": "-04:00",
"gte": "2022-04-17T06:00:00"
"time_zone": "-04:00",
"gte": "2022-04-17T06:00:00"
}
}
}
}
```
{% include copy-curl.html %}

The `gte` parameter in the preceding query is converted to `2022-04-17T10:00:00 UTC`, which is the UTC equivalent of `2022-04-17T06:00:00-04:00`.
The `gte` parameter in the preceding query is converted to `2022-04-17T10:00:00 UTC`, which is the UTC equivalent of `2022-04-17T06:00:00-04:00`.

The `time_zone` parameter does not affect the `now` value because `now` always corresponds to the current system time in UTC.
{: .note}
Expand All @@ -200,7 +200,7 @@ GET _search
"range": {
"<field>": {
"gt": 10,
...
...
}
}
}
Expand All @@ -215,7 +215,7 @@ Parameter | Data type | Description
:--- | :--- | :---
`format` | String | A [format]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/date/#formats) for dates in this query. Default is the field's mapped format.
`relation` | String | Indicates how the range query matches values for [`range`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/range/) fields. Valid values are:<br> - `INTERSECTS` (default): Matches documents whose `range` field value intersects the range provided in the query. <br> - `CONTAINS`: Matches documents whose `range` field value contains the entire range provided in the query. <br> - `WITHIN`: Matches documents whose `range` field value is entirely within the range provided in the query.
`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
`time_zone` | String | The time zone used to convert [`date`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/date/) values to UTC in the query. Valid values are ISO 8601 [UTC offsets](https://en.wikipedia.org/wiki/List_of_UTC_offsets) and [IANA time zone IDs](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). For more information, see [Time zone](#time-zone).

If [`search.allow_expensive_queries`]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries) is set to `false`, range queries on [`text`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/text/) and [`keyword`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/keyword/) fields are not run.
Expand Down
4 changes: 2 additions & 2 deletions _query-dsl/term/regexp.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ GET _search
"regexp": {
"<field>": {
"value": "[Ss]ample",
...
...
}
}
}
Expand All @@ -56,11 +56,11 @@ The `<field>` accepts the following parameters. All parameters except `value` ar
Parameter | Data type | Description
:--- | :--- | :---
`value` | String | The regular expression used for matching terms in the field specified in `<field>`.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
`case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the regular expression value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping).
`flags` | String | Enables optional operators for Lucene’s regular expression engine.
`max_determinized_states` | Integer | Lucene converts a regular expression to an automaton with a number of determinized states. This parameter specifies the maximum number of automaton states the query requires. Use this parameter to prevent high resource consumption. To run complex regular expressions, you may need to increase the value of this parameter. Default is 10,000.
`rewrite` | String | Determines how OpenSearch rewrites and scores multi-term queries. Valid values are `constant_score`, `scoring_boolean`, `constant_score_boolean`, `top_terms_N`, `top_terms_boost_N`, and `top_terms_blended_freqs_N`. Default is `constant_score`.

If [`search.allow_expensive_queries`]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries) is set to `false`, `regexp` queries are not run.
{: .important}

6 changes: 3 additions & 3 deletions _query-dsl/term/term.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ GET _search
"term": {
"<field>": {
"value": "sample",
...
...
}
}
}
Expand All @@ -95,5 +95,5 @@ The `<field>` accepts the following parameters. All parameters except `value` ar
Parameter | Data type | Description
:--- | :--- | :---
`value` | String | The term to search for in the field specified in `<field>`. A document is returned in the results only if its field value exactly matches the term, with the correct spacing and capitalization.
`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`.
`case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping).
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
`case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping).
5 changes: 3 additions & 2 deletions _query-dsl/term/terms-set.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ GET _search
"terms_set": {
"<field>": {
"terms": [ "term1", "term2" ],
...
...
}
}
}
Expand All @@ -167,4 +167,5 @@ Parameter | Data type | Description
:--- | :--- | :---
`terms` | Array of strings | The array of terms to search for in the field specified in `<field>`. A document is returned in the results only if the required number of terms matches the document's field values exactly, with the correct spacing and capitalization.
`minimum_should_match_field` | String | The name of the [numeric]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/numeric/) field that specifies the number of matching terms required in order to return a document in the results.
`minimum_should_match_script` | String | A script that returns the number of matching terms required in order to return a document in the results.
`minimum_should_match_script` | String | A script that returns the number of matching terms required in order to return a document in the results.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
5 changes: 3 additions & 2 deletions _query-dsl/term/terms.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The query accepts the following parameters. All parameters are optional.
Parameter | Data type | Description
:--- | :--- | :---
`<field>` | String | The field in which to search. A document is returned in the results only if its field value exactly matches at least one term, with the correct spacing and capitalization.
`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.

## Terms lookup

Expand Down Expand Up @@ -249,4 +249,5 @@ Parameter | Data type | Description
`index` | String | The name of the index in which to fetch field values. Required.
`id` | String | The document ID of the document from which to fetch field values. Required.
`path` | String | The name of the field from which to fetch field values. Specify nested fields using dot path notation. Required.
`routing` | String | Custom routing value of the document from which to fetch field values. Optional. Required if a custom routing value was provided when the document was indexed.
`routing` | String | Custom routing value of the document from which to fetch field values. Optional. Required if a custom routing value was provided when the document was indexed.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
2 changes: 1 addition & 1 deletion _query-dsl/term/wildcard.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ The `<field>` accepts the following parameters. All parameters except `value` ar
Parameter | Data type | Description
:--- | :--- | :---
`value` | String | The wildcard pattern used for matching terms in the field specified in `<field>`.
`boost` | Floating-point | Boosts the query by the given multiplier. Useful for searches that contain more than one query. Values in the [0, 1) range decrease relevance, and values greater than 1 increase relevance. Default is `1`.
`boost` | Floating-point | A floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
`case_insensitive` | Boolean | If `true`, allows case-insensitive matching of the value with the indexed field values. Default is `false` (case sensitivity is determined by the field's mapping).
`rewrite` | String | Determines how OpenSearch rewrites and scores multi-term queries. Valid values are `constant_score`, `scoring_boolean`, `constant_score_boolean`, `top_terms_N`, `top_terms_boost_N`, and `top_terms_blended_freqs_N`. Default is `constant_score`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ The OpenSearch 2.13.0 documentation includes the following additions and updates
- Add post_filter is supported in hybrid search [#6724](https://github.com/opensearch-project/documentation-website/pull/6724)
- Update deb/rpm autorestart service after upgrade documentation [#6720](https://github.com/opensearch-project/documentation-website/pull/6720)
- Add documentation page for Vega Visualizations [#6711](https://github.com/opensearch-project/documentation-website/pull/6711)
- Add cache plugin and tiered cache documentation [#6708](https://github.com/opensearch-project/documentation-website/pull/6708)
- Add documentation for text chunking processor [#6707](https://github.com/opensearch-project/documentation-website/pull/6707)
- Update documentation to support InnerProduct with k-NN Lucene Engine [#6703](https://github.com/opensearch-project/documentation-website/pull/6703)
- Add documentation for kuromoji_completion filter [#6699](https://github.com/opensearch-project/documentation-website/pull/6699)
Expand Down

0 comments on commit 3bb39c3

Please sign in to comment.