Skip to content

Run bravebot's security audit before the loop opens a bravebot pull request - #221

Merged
bbondy merged 1 commit into
masterfrom
bravebot-security-audit-before-pr
Nov 13, 2026
Merged

bbondy merged 1 commit into
masterfrom
bravebot-security-audit-before-pr

Conversation

@netzenbot

Copy link
Copy Markdown
Collaborator

Closes #219

User impact: a bravebot pull request the loop opens has passed bravebot's mechanical security audit first; a branch with a finding at error is fixed before it opens instead of reaching review.

The problem

bravebot has a check for a branch that is about to become a pull request: security-audit.py --before-pr, which audits what the branch changes against its rule that untrusted content never decides anything. When the loop works a bravebot story, nothing runs it, so a branch that breaks the rule is first seen by a reviewer, or not at all.

Reproduce

  1. In this checkout, with config.json on the bravebot profile, run ./run.sh 1 on any bravebot story.
  2. Read the acceptance criteria the story is given in data/prd.json.

On master the criteria end with the Docker gate step and never name the audit, so the pull request opens without it. Expected: a step that runs the audit after make check-affected and stops the pull request on a finding.

pytest tests/test_scripts.py -k test_audits_the_branch_before_the_pull_request_opens covers it. It fails on the parent commit and passes here.

The fix

The bravebot profile gets a validation step that runs python3 agents/skills/security-audit/security-audit.py --before-pr upstream/main once the host gates pass. The audit needs no model, so the step reads its exit status: 0 opens the pull request, 1 is a finding to fix first, and 2 means the step failed. projects/bravebot/docs/testing.md says what the check covers and what each exit code means. Other profiles are unchanged.

Test plan

  • pytest tests/test_scripts.py -k test_audits_the_branch_before_the_pull_request_opens passed. Shown failing on three wrong profiles: the step removed, the "do not open the pull request until it exits 0" wording removed, and the step moved before make check-affected. Each fails the test, and the restored profile passes.
  • ruff check . and ruff format --check . passed.
  • pytest tests/ passed, 1247 tests.
  • make check-reviewdog not run.
  • CI passes cleanly

The bravebot profile gains a validation step, after make check-affected, that runs python3 agents/skills/security-audit/security-audit.py --before-pr upstream/main. Exit 0 opens the pull request; exit 1 is a finding to fix first; exit 2 is a failed step. projects/bravebot/docs/testing.md says what the check does, and a test pins the step, its place after the host gates, and that a finding stops the pull request.

Part of brave/bravebot#3464
@bbondy
bbondy merged commit 715eaef into master Nov 13, 2026
3 checks passed
@bbondy
bbondy deleted the bravebot-security-audit-before-pr branch November 13, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run security-audit.py --before-pr before opening a pull request on brave/bravebot

2 participants