design(mind): PageIndex-style vectorless tree navigation — context and decisions

[keli-wen]

Context

We want a decided design for PageIndex-style, vectorless / reasoning-based
navigation over long financial documents (10-K/10-Q, prospectuses, research
reports) before writing code. This issue captures the investigation and frames
the design PR; it does not add implementation.

What PageIndex is. VectifyAI/PageIndex
(MIT) reframes retrieval as relevance classification performed by the model
over a document's real structure instead of nearest-neighbor lookup in
embedding space. A document becomes a table-of-contents-like tree (nodes with
title, id, page range, summary, children); a query is answered by an LLM that
reads the tree of titles/summaries, picks a branch, drills down, and lazily
loads leaf text. No embeddings, no chunking, no vector store on the hot path.
It addresses failure modes of vector RAG on structured documents: similarity ≠
relevance, chunk fragmentation, cross-reference blindness ("see Item 7A"), and a
stateless retriever that cannot use prior reasoning.

Current state on master

The direction is already anticipated in the codebase; only the operation is
missing:

• quantmind.knowledge documents PageIndex-style reasoning as the explicit
  purpose of TreeKnowledge (_tree.py): read root + children summaries, pick a
  branch, drill down, lazy-load leaf content; embeddings are a coarse pre-filter,
  never a replacement.
• contexts/design/library/local.md has a ## PageIndex Boundary;
  contexts/design/rag/document.md has ## Collection Search and PageIndex.
• After the source-first Paper Flow V1 refactor (refactor(paper): add source-first Paper Flow V1 #120), PaperFlowResult has
  no tree; Paper is not a TreeKnowledge. Any navigation tree must be a
  new, independently versioned artifact linked to an exact source, not a
  revived PaperTree.

Prior art we can reuse

• VectifyAI/PageIndex (MIT): TOC detection, recursive node subdivision, the
  node schema, and two retrieval paths (single-pass node selection and an
  OpenAI Agents SDK tool loop). Reuse subject to license/dependency/contract
  review.
• Internal FinMind / SmartAsk validated a two-level (document → page) navigation
  with two-step model selection and measured gains over an embedding baseline on
  FinanceBench. We generalize that to a verified multi-level tree with exact
  page-range citations.

Requirements to bake into the design

1. Multi-model compatibility. master passes cfg.model as a plain string
   to agents.Agent(...) with no provider detection, and the only embedding
   provider is OpenAI. The design must route all generation through one
   provider-agnostic model-resolution seam (Agents SDK LiteLLM integration) and
   keep the hybrid pre-filter behind the library's existing _EmbeddingProvider
   Protocol — no provider registry.
2. Hybrid search compatibility. Support "semantic search to shortlist nodes,
   then agentic reasoning over that shortlist." Because library already emits
   one projection per non-root TreeKnowledge node, search() returns candidate
   node_ids that can seed agentic navigation with no second index.

Relationship to existing issues

• Refines feat: page-preserving TreeKnowledge build + agentic navigation #95 (feat: page-preserving TreeKnowledge build + agentic navigation), reconciling its TreeKnowledge-based paper assumption with the
  post-refactor(paper): add source-first Paper Flow V1 #120 source-first model.
• Depends on Tracking: Migrate Agent layer to OpenAI Agents SDK #71 (mind scaffold + import-linter boundary). If unlanded,
  the implementation adds the minimal mind package enforcing
  mind -> library -> knowledge.
• Builds on feat(library): local semantic knowledge library MVP #111 (LocalKnowledgeLibrary, closed) and refactor(paper): add source-first Paper Flow V1 #120 (Paper Flow
  V1).

Deliverable

A design PR adding contexts/design/mind/navigation.md (ownership split,
TreeKnowledge-based tree model with Citation.end_page, deterministic-vs-model
build pipeline, single-pass + agentic navigation, the multi-model and hybrid
seams, import-linter placement, and phasing), plus cross-links from the existing
library and RAG boundary sections.