docs: add HNSW replication doc

[BorysTheDev]

Summary: Adds a design/spec document describing how Dragonfly replicates global HNSW vector indices during full sync.

Changes:

• Defines the data model (shard-local DocId, encoded GlobalDocId, and global graph vs per-shard key index).
• Specifies the RDB wire format: AUX fields (e.g. search-index, hnsw-index-metadata, shard-count) and custom opcodes for graph/mapping dumps.
• Documents index states (kProhibit/kRestoring/kSerializing/kBuilding) and invariants used to keep dumps consistent under concurrent writes.
• Describes master-side dump ordering (AUX → mappings → shard-0 graph dump → drain → key stream) and replica-side reconciliation/hydration.
• Explains how shard-count mismatches trigger a remap of GlobalDocIds before installing the graph.
• Notes version gating via VER6 capability and fallback behavior for older replicas / missing mappings.

[augmentcode]

🤖 Augment PR Summary

Summary: Adds a design/spec document describing how Dragonfly replicates global HNSW vector indices during full sync.

Changes:

• Defines the data model (shard-local DocId, encoded GlobalDocId, and global graph vs per-shard key index).
• Specifies the RDB wire format: AUX fields (e.g. search-index, hnsw-index-metadata, shard-count) and custom opcodes for graph/mapping dumps.
• Documents index states (kProhibit/kRestoring/kSerializing/kBuilding) and invariants used to keep dumps consistent under concurrent writes.
• Describes master-side dump ordering (AUX → mappings → shard-0 graph dump → drain → key stream) and replica-side reconciliation/hydration.
• Explains how shard-count mismatches trigger a remap of GlobalDocIds before installing the graph.
• Notes version gating via VER6 capability and fallback behavior for older replicas / missing mappings.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 4 suggestions posted.

Comment augment review to trigger a new review at any time.

[Copilot]

Pull request overview

Adds a new design/spec document describing how Dragonfly replicates global HNSW vector indices (graph + per-shard key→DocId mappings) via the RDB full-sync stream, including shard-count mismatch handling.

Changes:

• Documented replication data model (GlobalDocId), wire format (AUX fields + opcodes 222/223), and restore protocol.
• Defined shard-local HNSW lifecycle states/invariants relevant to concurrent writes during serialization/restore.
• Specified a remap procedure for differing master/replica shard counts and version gating via VER6.

[vyavdoshenko]

lgtm

[romange]

The explanations here are not clear

[romange]

i strongly suggest introducing diagrams: either excalidraw or mermaid - see other docs how they do it.

[romange]

LGTM