Fix GPU and Java-heap OOM when building large vector indexes - #2476
Open
nvzm123 wants to merge 2 commits into
Open
Fix GPU and Java-heap OOM when building large vector indexes#2476nvzm123 wants to merge 2 commits into
nvzm123 wants to merge 2 commits into
Conversation
Signed-off-by: Zack Meeks <zmeeks@nvidia.com>
nvzm123
marked this pull request as ready for review
August 20, 2026 04:03
rapids-bot Bot
pushed a commit
that referenced
this pull request
Sep 10, 2026
This PR replaces [cuVS-Lucene #195](NVIDIA/cuvs-lucene#195) as cuVS-Lucene has been merged into cuVS. This PR builds on the out-of-core host-streaming refactor from #2476 by @nvzm123. This PR subsumes #2476 to avoid stacking the PRs. On top of #2476, this PR adds the following. **Improvements** - Native flat buffering: stream vectors directly into a native host matrix during indexing instead of buffering them as a heap List<float[]>, cutting peak host memory from ~2x to ~1x and eliminating the per-vector matrix-assembly copy. Opt-in; requires all input vectors to be indexed in the original order, i.e. not supported for sorted, merged, or filtered index segments; not currently supported for segments with quantized fields. - Parallelized CAGRA-to-HNSW graph conversion: materializing the CAGRA adjacency into on-heap NeighborArrays was a serial per-node loop; parallelized under the existing `writerThreads` knob. - Parallelized level-0 HNSW graph serialization: level-0 (all N nodes) is now delta/VInt-encoded in parallel across memory-bounded waves of threads, then concatenated to the `IndexOutput` in node order according to on-disk format in the serial path. **New example: OptimizedCagraHnswBuildExample** A reference pattern for building a large accelerated HNSW index with all ingest- and build-side optimizations, showcasing: - Streaming, prefetched, bounded-memory ingestion: open the source file once, read it front-to-back in large sequential chunks; hold at most two chunks in memory; fill the next chunk on a background thread while the ingest thread drains the current one, hiding disk read behind indexing; unpack into a caller-reused float[] allocation (no per-vector allocation, safe because Lucene copies the value eagerly inside `addDocument`). - Native flat buffering sized exactly to each segment's vector count via `withNumInputVectors`, avoiding the heap-buffered assembly copy. - Overlapping multi-segment build: - Default: K single-segment passes appended to one directory; peak host memory is one slice (N/K). - Overlapped: a bounded pool (`PIPELINE_DEPTH`) builds segments concurrently into their own directories with the GPU commit serialized on a semaphore (ingest overlaps a prior segment's GPU commit), then combines the finished per-segment indexes by hardlinking their files into the final directory via `HardlinkCopyDirectoryWrapper` + `addIndexes` (no bulk copy of vector data). Peak host memory is up to `PIPELINE_DEPTH * N / K`. - An `enableRMMAsyncMemory()` call (with a note that it must not be used with CPU-only codecs) to opt-in to RMM-managed memory resources, plus exposing the primary tuning knobs (withMaxConn, withBeamWidth, withCuvsDistanceType, withWriterThreads). Authors: - James Xia (https://github.com/jamxia155) - https://github.com/nvzm123 - Igor Motov (https://github.com/imotov) Approvers: - James Lamb (https://github.com/jameslamb) - Igor Motov (https://github.com/imotov) URL: #2481
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This ports NVIDIA/cuvs-lucene#173 into
java/cuvs-lucenefollowing the move into the cuVS monorepo.It addresses two memory-pressure problems when building accelerated HNSW indexes over large datasets:
List<float[]>copy on the Java heap.The change also consolidates the HNSW graph and field-writing paths around
CuVSMatrix, removes redundant size/resource parameters, and derives quantized-vector byte width from the matrix so binary and scalar quantization use the correct width.The original work superseded NVIDIA/cuvs-lucene#141. The related increase of
MAX_HNSW_LAYERSto 99 is already present on the current monorepomainand is therefore not part of this diff.Validation status
Fresh port-only structural checks:
cd java/cuvs-lucene && mvn -q -DskipTests compile— passedcd java/cuvs-lucene && mvn -q spotless:check— passedThese commands compile and check formatting; they do not constitute behavioral or GPU validation.
The source PR historically reported local passes for:
TestMerge, includingtestLargeScaleMergewith-DlargeScale=trueTestCuVSVectorsFormatTestLucene99AcceleratedHNSWVectorsFormatTestQuantizedVectorsFormatsThose results predate this port and are not validation of the current branch. The original PR's final standalone-repository CI run was not green, so the relevant Java/GPU suites must be rerun from
java/cuvs-lucene. We also need combined validation with PR #2475, including its full PyLucene GPU end-to-end suite.The source PR also recorded an intermittent, seed-dependent
TestLucene99AcceleratedHNSWVectorsFormat.testCheckIntegrityReadsAllBytesfailure during full-suite randomized graph testing. It passed in isolation and reportedly occurred at a similar rate on the source base branch. This historical observation has not yet been re-evaluated in the monorepo.