[codex] Support Voyage contextual embeddings in pipelines

[williambarberjr]

Summary

• add voyage-context-3 support through VoyageAIEmbeddings, backed by Catsu
• add document-group embedding support to CatsuEmbeddings for contextual models
• route opted-in pipeline handshakes through write_documents, and make Turbopuffer preserve document boundaries before embedding chunks

Dependency

• depends on Catsu contextual embedding support in Support Voyage contextual embeddings catsu#32 before this can be released against a published catsu version

Test plan

• mocked Catsu and Voyage contextual embedding paths
• Turbopuffer document-boundary write path
• pipeline sync/async document-aware write routing
• Ruff check and format check on touched files
• live Voyage smoke: Catsu and Voyage wrappers returned three document vectors and 256-dim query vectors

Notes

• once Catsu PR feat(embeddings): Add Jina AI Embeddings support #32 is released, the Chonkie dependency lower bound can be tightened to the released version if desired

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d829f5ea-5d06-4193-9102-0396da877e64

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request transitions VoyageAI embeddings to the catsu provider and introduces support for contextualized models, specifically voyage-context-3. It adds embed_documents functionality to preserve document boundaries and updates the TurbopufferHandshake and Pipeline to support document-aware writing. Feedback suggests removing a redundant warning for the truncation parameter in VoyageAIEmbeddings and optimizing the hasattr check for contextualized embedding support in CatsuEmbeddings by moving it to the initialization phase.

[gemini-code-assist]

This specific warning for contextualized models is redundant. A more general warning for the truncation parameter is already issued at line 112 for all models in this class. Since VoyageAIEmbeddings ignores the truncation parameter for all models (as it is not supported by the underlying CatsuEmbeddings), the general warning is sufficient and covers this case.

[gemini-code-assist]

Performing a hasattr check on every call to _contextualized_embed is inefficient. Since the availability of this method depends on the installed version of the catsu package, it should be checked once during the initialization of the CatsuEmbeddings class when _is_contextualized is true.