Skip to content

fix(llm): make rerank context size configurable via QMD_RERANK_CONTEXT_SIZE#369

Open
mvanhorn wants to merge 1 commit intotobi:mainfrom
mvanhorn:osc/291-rerank-context-size-configurable
Open

fix(llm): make rerank context size configurable via QMD_RERANK_CONTEXT_SIZE#369
mvanhorn wants to merge 1 commit intotobi:mainfrom
mvanhorn:osc/291-rerank-context-size-configurable

Conversation

@mvanhorn
Copy link
Contributor

Fixes #291

Summary

Makes RERANK_CONTEXT_SIZE configurable via the QMD_RERANK_CONTEXT_SIZE environment variable, defaulting to 2048 for backwards compatibility. This follows the existing pattern used by QMD_EXPAND_CONTEXT_SIZE.

CJK content and long query expansions can exceed the 2048 default - the truncation logic (added previously) prevents crashes but aggressively truncates content. Users can now set QMD_RERANK_CONTEXT_SIZE=4096 to avoid truncation on CJK corpora.

Changes

  • Read QMD_RERANK_CONTEXT_SIZE from environment, parse as integer
  • Validate range (512-32768), fall back to 2048 on invalid values
  • Updated comment to document the env var

Test plan

  • QMD_RERANK_CONTEXT_SIZE=4096 qmd query "test" uses 4096 context
  • Default behavior unchanged without env var
  • Invalid values (negative, non-numeric) fall back to 2048
  • npm run build passes
  • npx vitest run test/ passes

This contribution was developed with AI assistance (Claude Code).

…T_SIZE

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RERANK_CONTEXT_SIZE (2048) too small — qmd query crashes on CJK content

1 participant