feat: implement new_session:true support for multi-turn task session …

[HaotianChen616]

feat: implement full multi-turn user prompt support with session isolation

Summary

PinchBench currently has partial multi-session support — the sessions field in task frontmatter is processed to send sequential prompts, but the new_session: true flag (already defined in task_second_brain.md) is completely ignored by the Python codebase. This means tasks that need to test cross-session memory or fresh-context behavior cannot work correctly.

This PR implements full multi-turn user prompt support by:

1. Implementing new_session: true session isolation in lib_agent.py — when a session entry has new_session: true, the agent's session state is cleaned up and a new session_id is generated, simulating a user returning after closing the agent. The workspace (and any files created) is preserved across sessions.

2. Adding transcript archiving per session — before starting a new session, the current session's transcript is archived. After all sessions complete, transcripts are merged so the grading engine can evaluate the full conversation history.

3. Documenting multi-session tasks in TASK_TEMPLATE.md — adds a complete "Multi-Session Tasks" section with field descriptions, usage guidelines, and YAML examples.

4. Adding two new multi-turn tasks:

   • task_iterative_code_refine.md — tests iterative code refinement across 3 sessions, with the final session using new_session: true to verify the agent can work from file-based context alone
   • task_session_chain_analysis.md — tests 4-turn structured code analysis (single-session multi-turn) where the agent reads TypeScript source files, produces JSON chain maps, designs minimal changes, extracts code evidence, and writes a delivery summary

5. Adding tests — test_multi_session.py covers frontmatter parsing, new_session flag handling, transcript archiving, and task loading.

Changes

scripts/lib_agent.py

• Added _archive_transcript() helper to save per-session transcripts before session resets
• Modified execute_openclaw_task() multi-session loop to:
  • Track current_session_id separately (changes when new_session: true)
  • Call cleanup_agent_sessions() and generate a new session ID when new_session: true
  • Log session transitions clearly
• After all sessions complete, merge archived transcripts with the final session transcript when new_session was used

tasks/TASK_TEMPLATE.md

• Added sessions and new_session YAML examples to the frontmatter template
• Added "Multi-Session Tasks" documentation section with field table, how-it-works explanation, and usage guidelines
• Added multi-session checklist items

tasks/task_iterative_code_refine.md (new)

• 3-session task: initial implementation → add error handling → fresh-session review
• Tests iterative refinement and cross-session file-based context retrieval
• Automated grading checks both calculator.py and review.txt

tasks/task_session_chain_analysis.md (new)

• 4-session multi-turn task (single conversation context, no new_session): structured ingest → minimal design → evidence extraction → delivery summary
• Source files from the OpenClaw agent command pipeline (session.ts, session-store.ts, delivery.ts, agent-command.ts) copied to tasks/assets/session_chain/
• Tests the agent's ability to maintain context across turns, produce structured JSON, extract precise code evidence, and generate traceable design review artifacts
• Automated grading checks chain_map stages, evidence_index, function references, JSON validity, and traceability

tasks/assets/session_chain/ (new)

• session.ts — OpenClaw session resolution logic (resolveSession, resolveSessionKeyForRequest)
• session-store.ts — Session store update logic (updateSessionStoreAfterAgentRun)
• delivery.ts — Agent command delivery logic (deliverAgentCommandResult)
• agent-command.ts — Agent command execution pipeline (prepareAgentCommandExecution, agentCommandInternal)

tasks/manifest.yaml

• Added task_iterative_code_refine and task_session_chain_analysis to the task list

tests/test_multi_session.py (new)

• TestMultiSessionFrontmatterParsing — sessions list, new_session flag, string entries
• TestNewSessionHandling — archive-before-cleanup behavior verification
• TestArchiveTranscript — transcript archiving with/without transcript path
• TestMultiSessionTaskLoading — integration test verifying task_second_brain and task_iterative_code_refine load correctly

Motivation

Real-world AI coding agents are used in multi-turn conversations. Users:

• Ask agents to create something, then refine it iteratively
• Return to agents in new sessions expecting them to pick up where they left off using files/context
• Need agents that can work with both conversation history AND file-based persistence

PinchBench should test all of these scenarios. The existing task_second_brain.md was designed to test cross-session memory, but it couldn't work correctly because new_session: true was never implemented.

Testing

# Run the new tests
cd /path/to/skill
uv run pytest tests/test_multi_session.py -v

# Lint check
ruff check scripts/lib_agent.py

Backward Compatibility

• Single-session tasks: No change in behavior
• Multi-session tasks without new_session: No change in behavior (all prompts share one session, same as before)
• Multi-session tasks with new_session: true: Now correctly isolates sessions. Previously, new_session was silently ignored, so this is a bug fix that makes task_second_brain.md work as originally intended.

[HaotianChen616]

#329

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

This PR has evolved through several iterations — the final state adds categorized manifest support, per-category score rollups in benchmark output, improved judge response parsing, and new multi-turn tasks. All of it is well-implemented with good defensive error handling.

Files Reviewed

• scripts/benchmark.py — category filtering in _select_task_ids, extracted _compute_category_scores, category scores in results JSON
• scripts/lib_tasks.py — categorized manifest format (category_map, categories, run_first)
• scripts/lib_grading.py — improved judge response parsing with reverse-scan, new _coerce_score_value / _extract_named_scores helpers
• scripts/lib_upload.py — category field and category_scores forwarded to upload payload
• scripts/lint_manifest.py — validation for categorized manifest format
• scripts/lib_agent.py — reverted new_session changes (back to baseline)
• tasks/manifest.yaml — migrated to categorized format
• Various task files and assets
• tests/test_lib_grading.py — new tests for judge response parsing

---

_{Reviewed by claude-4.6-sonnet-20260217 · 551,514 tokens}

[ScuttleBot]

👋 Hi @HaotianChen616! I'm @olearycrew's OpenClaw bot doing a triage pass.

Heads up — this PR now has merge conflicts with main, likely from the batch of task PRs that were merged yesterday (#315, #319, #320, #322, and others all touched manifest.yaml).

Would you be able to rebase on main when you get a chance? The Kilo Code Review still shows no issues and recommends merge, so once conflicts are resolved this should be good to go.

Thanks for implementing the new_session:true support! 🦀

[HaotianChen616]

👋 Hi @HaotianChen616! I'm @olearycrew's OpenClaw bot doing a triage pass.

Heads up — this PR now has merge conflicts with main, likely from the batch of task PRs that were merged yesterday (#315, #319, #320, #322, and others all touched manifest.yaml).

Would you be able to rebase on main when you get a chance? The Kilo Code Review still shows no issues and recommends merge, so once conflicts are resolved this should be good to go.

Thanks for implementing the new_session:true support! 🦀

Of course — thanks for the heads up! I'll rebase on main to resolve the merge conflicts and follow up once that's done.

[HaotianChen616]

👋 Hi @HaotianChen616! I'm @olearycrew's OpenClaw bot doing a triage pass.

Heads up — this PR now has merge conflicts with main, likely from the batch of task PRs that were merged yesterday (#315, #319, #320, #322, and others all touched manifest.yaml).

Would you be able to rebase on main when you get a chance? The Kilo Code Review still shows no issues and recommends merge, so once conflicts are resolved this should be good to go.

Thanks for implementing the new_session:true support! 🦀

I've rebased on main and resolved the conflict. The PR should now be conflict-free and ready to merge.

[HaotianChen616]

@olearycrew Rebased on main and resolved the merge conflicts. Should be good to go now — CI and code review both look clean. Let me know if anything else is needed.

[olearycrew]

@HaotianChen616 thanks for this contribution!