You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/elasticsearch/mapping-reference/dense-vector.md
+3-14Lines changed: 3 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -234,7 +234,7 @@ $$$dense-vector-similarity$$$
234
234
`l2_norm`
235
235
: Computes similarity based on the L2 distance (also known as Euclidean distance) between the vectors. The document `_score` is computed as `1 / (1 + l2_norm(query, vector)^2)`.
236
236
237
-
For `bit` vectors, instead of using `l2_norm`, the `hamming` distance between the vectors is used. The `_score` transformation is `(numBits - hamming(a, b)) / numBits`
237
+
For `bit` vectors, instead of using `l2_norm`, the `hamming` distance between the vectors is used. The `_score` transformation is `(numBits - hamming(a, b)) / numBits`
238
238
239
239
`dot_product`
240
240
: Computes the dot product of two unit vectors. This option provides an optimized way to perform cosine similarity. The constraints and computed score are defined by `element_type`.
: (Required, string) The type of kNN algorithm to use. Can be either any of:
268
268
* `hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) for scalable approximate kNN search. This supports all `element_type` values.
269
269
* `int8_hnsw` - The default index type for some float vectors:
270
-
271
270
* {applies_to}`stack: ga 9.1` Default for float vectors with less than 384 dimensions.
272
271
* {applies_to}`stack: ga 9.0` Default for float all vectors.
273
-
274
272
This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 4x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
275
273
* `int4_hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 8x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
276
274
* `bbq_hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically binary quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 32x at the cost of accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
277
-
275
+
278
276
{applies_to}`stack: ga 9.1``bbq_hnsw` is the default index type for float vectors with greater than or equal to 384 dimensions.
279
277
* `flat` - This utilizes a brute-force search algorithm for exact kNN search. This supports all `element_type` values.
280
278
* `int8_flat` - This utilizes a brute-force search algorithm in addition to automatically scalar quantization. Only supports `element_type` of `float`.
: (Optional, float) Only applicable to `int8_hnsw`, `int4_hnsw`, `int8_flat`, and `int4_flat` index types. The confidence interval to use when quantizing the vectors. Can be any value between and including `0.90` and `1.0` or exactly `0`. When the value is `0`, this indicates that dynamic quantiles should be calculated for optimized quantization. When between `0.90` and `1.0`, this value restricts the values used when calculating the quantization thresholds. For example, a value of `0.95` will only use the middle 95% of the values when calculating the quantization thresholds (e.g. the highest and lowest 2.5% of values will be ignored). Defaults to `1/(dims + 1)` for `int8` quantized vectors and `0` for `int4` for dynamic quantile calculation.
292
290
293
-
294
291
`rescore_vector` {applies_to}`stack: preview 9.0, ga 9.1`
295
292
: (Optional, object) An optional section that configures automatic vector rescoring on knn queries for the given field. Only applicable to quantized index types.
`dense_vector` fields support [synthetic `_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) .
312
307
313
-
314
-
## Indexing & Searching bit vectors [dense-vector-index-bit]
308
+
## Indexing and searching bit vectors [dense-vector-index-bit]
315
309
316
310
When using `element_type: bit`, this will treat all vectors as bit vectors. Bit vectors utilize only a single bit per dimension and are internally encoded as bytes. This can be useful for very high-dimensional vectors or models.
317
311
@@ -354,7 +348,6 @@ PUT my-bit-vectors
354
348
355
349
1. The number of dimensions that represents the number of bits
356
350
357
-
358
351
```console
359
352
POST /my-bit-vectors/_bulk?refresh
360
353
{"index": {"_id" : "1"}}
@@ -366,7 +359,6 @@ POST /my-bit-vectors/_bulk?refresh
366
359
1. 5 bytes representing the 40 bit dimensioned vector
367
360
2. A hexidecimal string representing the 40 bit dimensioned vector
368
361
369
-
370
362
Then, when searching, you can use the `knn` query to search for similar bit vectors:
371
363
372
364
```console
@@ -412,7 +404,6 @@ POST /my-bit-vectors/_search?filter_path=hits.hits
412
404
}
413
405
```
414
406
415
-
416
407
## Updatable field type [_updatable_field_type]
417
408
418
409
To better accommodate scaling and performance needs, updating the `type` setting in `index_options` is possible with the [Update Mapping API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-mapping), according to the following graph (jumps allowed):
@@ -477,5 +468,3 @@ Vectors indexed before this change will keep using the `flat` type (raw float32
477
468
In order to have all the vectors updated to the new type, either reindexing or force merging should be used.
478
469
479
470
For debugging purposes, it’s possible to inspect how many segments (and docs) exist for each `type` with the [Index Segments API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments).
0 commit comments