fix(dspy): compute partial parse reward from field count ratio

[detail-app]

Warning

GitHub issue creation failed

Detail attempted to publish this bug to GitHub, but the issue could not be created. This fix PR was created without that issue, and missing tracker references are shown as Unknown issue.

You can review and merge this PR normally. Please review your tracker integration settings before the next publish run.

Detail bug report: View on Detail

📝 Changes Description

Closes Unknown issue

Bug

• bootstrap_trace_data's partial-parse handler divided two list[str] values (present / expected) instead of their lengths, raising TypeError: unsupported operand type(s) for /: 'list' and 'list' whenever an LM returned a parseable, non-empty dict that was missing one or more required (non-default, non-nullable) output fields.
• Because every shipping caller (GRPO, GEPA) runs with raise_on_error=False, the TypeError was swallowed by the evaluator's error budget and the affected example was silently dropped from the returned data (via the ValueError → continue path), producing no partial-credit FailedPrediction. GRPO/GEPA thus lost training signal on exactly the malformed-but-parseable responses the partial-credit path is meant to train on. The in-tree TODO at bootstrap_trace.py:136-137 documents this failure mode occurring in practice (Qwen/Qwen2.5-Coder-0.5B-Instruct on MATH).

Fix

• In dspy/teleprompt/bootstrap_trace.py, replace present / expected with len(present) / len(expected). The partial-parse reward now interpolates correctly: format_failure_score + (failure_score - format_failure_score) * (fraction of expected output fields present) — a numeric value between format_failure_score and failure_score. The example is retained as a FailedPrediction (with its completion text and trace) instead of being silently dropped.
• Added focused regression tests in tests/teleprompt/test_bootstrap_trace.py using a two-required-output-field signature (answer, reasoning) and a parseable response missing a required field ({"answer": "4"}).

Testing

• Unit tests, ruff lint, and ruff format all pass. The two new regression tests fail without the fix (reproducing the exact TypeError and silent drop) and pass with it. The full tests/teleprompt/ suite (90 tests, including the GRPO/GEPA callers of bootstrap_trace_data) and the CI-equivalent non-extra suite (pytest -m 'not extra and not deno' tests/, 1253 tests) pass with no regressions.
• Verified end-to-end against the real scenario from the in-tree TODO: ran bootstrap_trace_data against Qwen/Qwen2.5-Coder-0.5B-Instruct (via Ollama) on the MATH algebra split with a two-required-output-field signature. With truncated JSON (max_tokens=12) the model emits a partial dict missing reasoning; with the fix all 6 examples are retained as FailedPrediction with format_reward=-0.5 and zero list/list error logs, whereas without the fix all 6 are silently dropped with 6× ERROR ... 'list' and 'list' log lines. The happy-path control (max_tokens=400) retains all 6 as normal Predictions both before and after the fix.

✅ Contributor Checklist

• Pre-Commit checks are passing (locally and remotely)
• Title of your PR / MR corresponds to the required format
• Commit message follows required format fix(dspy): ...

⚠️ Warnings

Authored by Detail (automatic fix). The change is a one-line numeric correction plus two focused regression tests; the happy-path and the fully-unparseable (present is None) branches are unchanged.

---

Automatic Fixes PRs can be configured here.

[greptile-apps]

Greptile Summary

This PR fixes partial-parse reward calculation by dividing the number of parsed fields by the number of expected fields rather than attempting to divide two lists.

• Retains malformed-but-partially-parseable responses as FailedPrediction training examples.
• Adds regression coverage for default and caller-supplied reward ranges.
• Verifies completion text, score propagation, and trace preservation.

Confidence Score: 5/5

The PR appears safe to merge; the corrected calculation matches the adapter’s partial-result representation and is covered by focused tests.

No actionable failures remain: parsed fields are constrained to expected outputs, the denominator cannot be empty in the partial-result branch, and test-global configuration is restored automatically.

Important Files Changed

Filename	Overview
dspy/teleprompt/bootstrap_trace.py	Corrects the partial-parse interpolation formula to use a numeric field-count ratio.
tests/teleprompt/test_bootstrap_trace.py	Adds focused regression tests covering retention, reward interpolation, score propagation, and trace contents.

_{Reviews (1): Last reviewed commit: "fix(dspy): compute partial parse reward ..." | Re-trigger Greptile}