feat(guardrails): content policy at the five edges of a run - #122
Conversation
Adds timbal.guardrails: a verdict-based policy layer that runs at input, model_output, model_step, tool_args, and tool_result. A blocked input spends zero tokens and executes zero tools; a blocked tool call feeds the block back to the LLM so it can self-correct. Verdicts are allow | block | replace | retry | escalate | warn. escalate on tool_args reuses the existing HITL approval gate rather than inventing a second one. guardrail_mode="shadow" and per-rail sample_rate make rollout and online monitoring safe by default. Streaming is handled rather than sidestepped: deterministic redact rails scrub text and thinking deltas in flight behind a holdback window, and any rail that can block or retry buffers until it has a verdict, so no chunk escapes ahead of enforcement. Trace redaction scrubs spans on copies at store/export time, leaving the live run untouched while resumed sessions load redacted history. Built-ins: DetectPII, RedactSecrets, PromptInjection, KeywordGuard, MaxLength, Moderate, TopicGuard, LLMJudge. Rubrics (parse_rubric/grade_rubric) grade one isolated judge per criterion and drive both LLMJudge's grade-revise-regrade loop and a new rubric! eval validator. Also fixes a latent codegen defect the new transformers exposed: apply_operation imported every module in transformers/ to dispatch one, so a single bad import took down all twelve operations. Dispatch now loads only the requested module, and the shared guardrail CST helpers live in codegen/guardrail_specs.py instead of one transformer reaching into another's privates. The injection pattern pack is regression-fenced by a corpus of known attacks and benign lookalikes. Building it surfaced three real bugs, now fixed: newlines bypassed every pattern, and "print the instructions for the desk" / "remove the safety guard from my lawnmower" were false positives. Attacks the pack provably cannot catch (multilingual, base64, homoglyph) are xfail-documented rather than hidden — those need PromptInjection(model=...).
PR SummaryHigh Risk Overview Streaming and traces get first-class treatment: deterministic rails scrub text/thinking deltas with holdback; block/retry rails buffer until verdict; Codegen gains Docs add Reviewed by Cursor Bugbot for commit 2ef8138. Bugbot is set up for automated code reviews on this repo. Configure here. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Non-mutating rails were batched concurrently against the original stage text regardless of position, so a block/judge rail placed after a redactor judged pre-redaction content — and received the raw PII the redactor was put in front of it to strip. Rails now see the previous rail's output: adjacent non-rewriting rails still share one gather, and redact/retry rails are barriers. Batching is decided from the configured action, so a rail that returns replacement text without declaring action="redact" now raises with the fix in the message instead of becoming an invisible ordering bug. Dict (tool-arg) replacements are unaffected.
…rs actionable set-config's AGENT_FIELDS allowlist predated guardrails, so guardrails, guardrail_mode, and max_guardrail_retries were rejected outright — the agent-level knobs had no codegen path. They're allowlisted now and validated at the CLI: guardrail_mode must be enforce|shadow, and every entry of a guardrails list goes through coerce_rail so a typo fails with the valid names instead of at agent construction. The emitted literal list composes with add-/remove-guardrail edits, and tool-local rails work via set-config --name on Tool-wrapped tools. add-guardrail --spec now tells the truth about config-required rails: topic, judge, keywords, and length have required params a name[:action] shorthand cannot carry, so instead of dumping a pydantic validation error the CLI says so and points to code (Agent(guardrails=[TopicGuard(allow=[...])])). README documents the shorthand-ready vs code-only split and the set-config flows.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 91a1a14. Configure here.
… string presets at runtime Configuring a tool by name only ran field-name checks, so invalid guardrail shorthands reached Tool(...) and failed at agent construction rather than at the CLI. Tool configs now share the agent path's validation. "default" is only a preset as the whole value; as a list entry it reaches coerce_rail as an unknown shorthand. Reject it at the boundary and point at the two spellings that work. Tool(guardrails="default") iterated the string per character, so the runtime died on shorthand 'd' before the preset was ever expanded. Treat a string guardrails value as one spec and expand the preset before merging with agent-level rails.
2 participants
Adds timbal.guardrails: a verdict-based policy layer that runs at input, model_output, model_step, tool_args, and tool_result. A blocked input spends zero tokens and executes zero tools; a blocked tool call feeds the block back to the LLM so it can self-correct.
Verdicts are allow | block | replace | retry | escalate | warn. escalate on tool_args reuses the existing HITL approval gate rather than inventing a second one. guardrail_mode="shadow" and per-rail sample_rate make rollout and online monitoring safe by default.
Streaming is handled rather than sidestepped: deterministic redact rails scrub text and thinking deltas in flight behind a holdback window, and any rail that can block or retry buffers until it has a verdict, so no chunk escapes ahead of enforcement. Trace redaction scrubs spans on copies at store/export time, leaving the live run untouched while resumed sessions load redacted history.
Built-ins: DetectPII, RedactSecrets, PromptInjection, KeywordGuard, MaxLength, Moderate, TopicGuard, LLMJudge. Rubrics (parse_rubric/grade_rubric) grade one isolated judge per criterion and drive both LLMJudge's grade-revise-regrade loop and a new rubric! eval validator.
Also fixes a latent codegen defect the new transformers exposed: apply_operation imported every module in transformers/ to dispatch one, so a single bad import took down all twelve operations. Dispatch now loads only the requested module, and the shared guardrail CST helpers live in codegen/guardrail_specs.py instead of one transformer reaching into another's privates.
The injection pattern pack is regression-fenced by a corpus of known attacks and benign lookalikes. Building it surfaced three real bugs, now fixed: newlines bypassed every pattern, and "print the instructions for the desk" / "remove the safety guard from my lawnmower" were false positives. Attacks the pack provably cannot catch (multilingual, base64, homoglyph) are xfail-documented rather than hidden — those need PromptInjection(model=...).