Commit 6dd52c6
committed
fix: pass questionDate into question checkpoint so answer prompts get the real date
The benchmark loaders (e.g. LongMemEval) put question_date into
question.metadata.questionDate, and the answer phase reads
checkpoint.questions[id].questionDate to fill the {{questionDate}} /
'Question Date:' slot in every answer prompt. But the orchestrator's
initQuestion call never copied questionDate from the loaded question
into the checkpoint, so the value was always undefined and every
answer prompt rendered 'Question Date: Not specified'.
Temporal-reasoning questions ('what did I buy 10 days ago?') are
unanswerable without the question date, so temporal category scores
were understated for every provider measured on the harness.1 parent 118209a commit 6dd52c6
1 file changed
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
242 | 242 | | |
243 | 243 | | |
244 | 244 | | |
| 245 | + | |
245 | 246 | | |
246 | 247 | | |
247 | 248 | | |
| |||
0 commit comments