Skip to content

benchmark-index.md

Latest commit

 

History

History
77 lines (60 loc) · 4.61 KB

benchmark-index.md

File metadata and controls

77 lines (60 loc) · 4.61 KB
Raw

Benchmark Packet Index

This page is a directory of packet-backed benchmark surfaces.

Use benchmarks.md for the current benchmark tables and selected results. Use Benchmark Reporting Standard for the required fields in new AM, quantizer, storage-format, and option-set comparisons. Use this page when you need to find the underlying packet quickly.

Only link measured or intentionally scaffolded benchmark packets here. If a benchmark lane does not yet have a packet, leave it out of this index rather than inventing an empty result.

Current Full-Scale Evidence

The current production benchmark evidence (both AWS lanes, all four access methods × quantizations × 10K/50K/100K/1M) is the Task 105 full-scale matrix: reviews/task-105/006-full-scale-matrix/ (G4 + Intel), with per-lane raw evidence in reviews/task-105/004-g4-lane/ and 005-intel-lane/. Distributed multi-node SPIRE is reviews/task-107/. The benchmarks/current/<lane>/ lanes below are the promotion targets for that evidence and remain pending refresh.

Current Lanes

Lane Host class Current manifest Standard suite
M5 local Apple Silicon M5 local host benchmarks/current/m5-local/manifest.md crates/ecaz-cli/suites/current/m5-local.json
Intel local Local Intel desktop/workstation benchmarks/current/intel-local/manifest.md crates/ecaz-cli/suites/current/intel-local.json
AWS Intel AWS x86_64 benchmark host benchmarks/current/aws-intel/manifest.md crates/ecaz-cli/suites/current/aws-intel.json
AWS Graviton AWS arm64 benchmark host benchmarks/current/aws-graviton/manifest.md crates/ecaz-cli/suites/current/aws-graviton.json

DiskANN

Lane Corpus / fixture Packet
Initial local readiness cross-engine sweep real10K, isolated PG18 sweep reviews/task-29d/003-11109-task29d-final-readiness/
NEON exact rerank kernel A/B synth10K, real10K, real10K_w800 reviews/task-29/017-30204-task29-diskann-m5-neon-rerank/
Heap-TID rerank fetch A/B real10K_w800 reviews/task-29/018-30205-task29-diskann-m5-rerank-heap-order/
Heap-block prefetch warm-cache A/B real10K_w800 reviews/task-29/019-30206-task29-diskann-m5-rerank-prefetch/
M5 decision summary packet rollup only reviews/task-29/020-30207-task29-diskann-m5-decision/
Build-time scalar vs NEON A/B real10K reviews/task-29/021-30208-task29-diskann-m5-build-neon-followup/
Cold-cache prefetch 100K A/B real100K reviews/task-29/022-30209-task29-diskann-m5-cold-cache-100k/
Final post-M5 cross-engine refresh real10K, isolated PG18 sweep reviews/task-32/001-30210-task32-m5-diskann-final-cross-engine-refresh/

IVF

Lane Corpus / fixture Packet
Local landing status 10K / 25K / 100K / 990K directional summary reviews/task-28/140-30151-task28-ivf-local-landing-status/
100K current build 100K selected point reviews/task-28/113-30119-task28-ivf-a9-100k-current-build/
M5 current candidate decision 100K balanced + quality points reviews/task-31/038-30203-task31-current-m5-candidate-decision/

Quantizer And Storage Candidates

Lane Corpus / fixture Packet
IVF TurboQuant / PQ-FastScan / RaBitQ head-to-head 10K smoke reviews/task-28/082-30084-task28-ivf-quantizer-headtohead-smoke/
IVF current auto recommendation packet synthesis over 10K and 100K evidence reviews/task-28/094-30096-task28-ivf-current-auto-recommendation/
IVF RaBitQ A10 fill 10K / 25K local RaBitQ rows reviews/task-28/136-30144-task28-ivf-a10-rabitq-fill/
SPIRE RaBitQ remote endpoint gate PG18 loopback remote-serving readiness reviews/task-30/631-30709-spire-remote-heap-endpoint-gate/
SPIRE tuple transport measurement PG18 loopback, storage_format=rabitq benchmarks/30975-spire-tuple-transport-measurement/

HNSW

Lane Corpus / fixture Packet
Local reference row used in DiskANN comparison real10K, isolated PG18 sweep reviews/task-29d/003-11109-task29d-final-readiness/
M5 reference-refresh scaffold real50K, PG18 worker-count sweep plan reviews/task-33/001-30212-task33-hnsw-m5-reference-refresh/

Notes

  • Packet-local artifacts/manifest.md files are the source of truth for commands, SHAs, hardware, cache state, and cited result lines.
  • When adding a new benchmark packet, update this index and benchmarks.md together.
  • Candidate comparison packets should follow Benchmark Reporting Standard.