Skip to content

fix(rlm): honor odd truncation limits - #69

Draft
isaacbmiller wants to merge 1 commit into
review-base/stanford-main-10014from
isaac/rlm-exact-truncation
Draft

fix(rlm): honor odd truncation limits#69
isaacbmiller wants to merge 1 commit into
review-base/stanford-main-10014from
isaac/rlm-exact-truncation

Conversation

@isaacbmiller

Copy link
Copy Markdown

1. Issue / repro

Both RLM truncation paths promise a character limit but violate it when the limit is odd.

REPLVariable.from_value("value", "abcdef", preview_chars=1).preview
REPLEntry.format_output("abcdef", max_output_chars=1)

On main, both retain the full six-character value:

...abcdef

Output (6 chars):

... (5 characters omitted) ...

abcdef

The second result is especially misleading: it says five characters were omitted while displaying all six.

2. Why this is the root cause

Both implementations split the limit in half and reuse the floor-divided value for the head and tail:

half = limit // 2
head = value[:half]
tail = value[-half:]

For every positive odd limit, this retains limit - 1 source characters. At limit 1, half is zero and Python's value[-0:] is the complete string, so the cap fails entirely.

3. How we know the fix addresses the root cause

The regression cases cover both odd-number behaviors in both truncators:

  • limit 1: retain exactly the final character, never the complete value;
  • limit 5: retain two head characters and three tail characters.

The exact limit-1 assertions fail on untouched main at 24ec85de4, where the observed preview is ...abcdef. All four cases pass here.

4. Why this is the concise fix

The configured limit is divided once, and the remainder goes to the tail:

head_chars = limit // 2
tail_chars = limit - head_chars

That is the complete production change, applied at the two existing truncation sites. It does not introduce a helper, change marker text, validate new inputs, or alter prompt formatting.

5. Context needed to validate the change

The limit counts retained source characters; the existing ... or omission message is additional formatting. RLM intentionally preserves both the beginning and end of long values and outputs. Giving the odd remainder to the tail retains one more trailing character, where exception details and final results commonly appear.

6. What the fix does in the code

  • Replaces symmetric floor halves with explicit head and tail counts.
  • Makes their sum equal the configured positive limit.
  • Adds limit-1 and limit-5 regressions for REPLVariable previews.
  • Adds the same regressions for formatted REPLEntry output.

Compatibility boundaries and downsides

  • Even limits are byte-for-byte unchanged, including the defaults (1,000 and 10,000).
  • Positive odd limits intentionally change: prompts retain one additional trailing source character.
  • Limit 1 intentionally stops exposing the full value/output. Any snapshot relying on the buggy output will change.
  • Zero and negative limits are unchanged and remain outside this PR. Adding validation is a separate API/exception decision.
  • No API, type, provider, interpreter-lifecycle, or fallback change.
  • Merge sequencing: upstream PR #9801 edits the adjacent REPLEntry.format_output line for Markdown-fence safety. The changes are semantically independent, but if fix(rlm): protect history from nested fences stanfordnlp/dspy#9801 lands first this branch needs one trivial rebase conflict resolution.

Validation

  • uv run --frozen pytest -q tests/predict/test_rlm.py::TestREPLTypes --deno15 passed
  • uv run --frozen pytest -q tests/predict/test_rlm.py --deno113 passed, 2 skipped
  • repository pre-commit hooks on both changed files — passed
  • exact limit-1 assertion against untouched main — fails and prints the complete abcdef; passes here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant