fix(feedback): add transfer evidence cards (rebased on leak fix)#51
Draft
Abhishek21g wants to merge 9 commits into
Draft
fix(feedback): add transfer evidence cards (rebased on leak fix)#51Abhishek21g wants to merge 9 commits into
Abhishek21g wants to merge 9 commits into
Conversation
This contradicts the project's own stated contract — README ("private/ ...
never exposed to the agent") and docs/walkthrough.md ("the LLM is not told about
data/private/ ... prevents the agent from cheating"). It is reproducible on a
bundled task: the `longcot-chess` grader writes results.json as
`{"summary": {...}, "results": [{"expected": <correct answer>, ...}]}`, and
`_build_feedback_context` dumps the whole file — every held-out answer — into the
feedback-agent prompt via `json.dumps(eval_data)`. The feedback agent (told to
improve the target agent, and able to read the filesystem) can then optimize
toward the answer key instead of toward a genuinely better solution — a
reward-hacking / contamination surface. (lawbench exposes only aggregate
per-class accuracy; gpqa writes its gold to a different filename and isn't
dumped — one leaking bundled task is enough.)
This fixes the leak with three composable, task-agnostic layers:
Layer 1 — curated, reference-answer-free eval summary.
`_build_feedback_context` no longer dumps results.json. It calls a new
`_build_eval_summary` that, via `_collect_scalars`, emits every SCALAR metric at
any nesting depth as a JSON object — preserving the original results.json shape
and field names (top-level `accuracy`/`correct`/`total`, or a nested `summary`
block; whatever the grader wrote). The reference-answer-free guarantee comes
from DROPPING every list: per-item record arrays (`results`/`details`, where the
bundled graders put the answer key) are excluded wholesale; nested dicts are
recursed for their scalars. The only per-item channel is the generic, opt-in
`items[]` array, surfaced solely through a fixed render whitelist
(id/status/group/category/input/output/detail) so no task-specific key can ride
along; when it carries failures, a capped sample (diversified across status and
group) plus an anti-reward-hack framing line are appended. A grader that emits
only scalars gets output equivalent to the original dump, minus answer-bearing
arrays. `summarize_items` also stores a bounded items digest in context.md.
Layer 2 — held-out-integrity prompt notice.
A generic, path-free instruction injected into the meta and feedback prompts
telling the agent that grader-only ground truth exists in a private directory
and must not be accessed.
Layer 3 — protected-path defense-in-depth (two complementary guards), keyed off
the same generic `compute_protected_paths(task_dir)`:
3a — claude-impl PreToolUse deny hook.
Denies any Bash/Read/Edit/Glob/... tool call resolving inside the task's
held-out dir (`<task_dir>/data/private`), wired via a new `protected_paths`
kwarg. Defaults to None (no hook), so behavior is byte-identical when unused.
The Bash check is a path-string match, and only the `claude` impl honors the
kwarg.
3b — impl-agnostic OS-level guard.
A `restricted_access` context manager strips all filesystem permissions on
`data/private` (chmod 000) for the duration of the sequential meta/feedback
agent run, restoring the original mode on exit (including on error). It wraps
only the agent calls — never grading, which legitimately reads the dir. Gated
by `SIA_PRIVATE_DIR_GUARD` (default on). 3b closes two gaps 3a leaves open:
it protects the held-out dir for the openai/pydantic-ai/openhands impls that
ignore the kwarg, and it fails the read at the filesystem layer, which an
obfuscated shell or subprocess cannot talk around. Residual caveats: a root
process or a copy-out-then-read still bypasses it, and a SIGKILL mid-run
leaves the dir stripped until restored — container sandboxing (--sandbox
docker) remains the strongest boundary.
Three config knobs (`SIA_VERIFIER_PASS_STATUSES`, `SIA_FEEDBACK_FAILURE_SAMPLES`,
`SIA_PRIVATE_DIR_GUARD`) are documented in docs/configuration.md; the first two
default to today's behavior, the third turns on the OS-level guard by default.
Tests cover the leak guard (reference-answer sentinel in results[] never
appears), the real nested-`summary`/`results[]` grader shape, scalar recursion,
the render whitelist, the protected-path matcher matrix, the OS-level guard
(strip/restore, no-op when disabled, restore-on-error, missing-path tolerance),
the prompt notice, and source-level genericity guards. Only the three prompt
goldens change (the Layer-2 notice); the feedback-context goldens are unchanged.
This was referenced Jul 6, 2026
Author
Abhishek21g
marked this pull request as draft
July 6, 2026 01:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add per-generation
transfer_evidence.jsonto separate reusable improvements from task-specific residue before the feedback handoff. Closes #38.This branch is rebased onto the #36 leak-fix changes (
pr/36-leak-fix), resolving conflicts in orchestrator tests and refreshing thecontext.mdgolden.Changes
transfer_evidence.jsonnext toresults.jsonafter each generationimprovement.mdbulletsTRANSFER EVIDENCEsection in feedback contextcontext.mdDependency
Best merged after #36 (held-out answer leak fix). This branch includes that fix in its history.
Test plan
python -m pytest tests/ -q— 184 passedpython -m ruff check sia/ tests/— cleantests/test_orchestrator_helpers.py,tests/test_context_manager.pytests/golden/context.mdrefreshed