-
Notifications
You must be signed in to change notification settings - Fork 1
Wave 1: eval-explain, shadow-mode, cli-distribution, editor-schemas, diff-fuzz-bench #4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 61 commits
Commits
Show all changes
85 commits
Select commit
Hold shift + click to select a range
422d19f
fix: regenerate SDK contracts for phase-gated guard rules
bb-connor 1d978d1
feat(cli): add h2h eval subcommand with decision exit codes
bb-connor 21c81e9
feat(cli): add content and args-size action flags to h2h eval
bb-connor 9f582ea
feat(cli): add origin and posture context flags to h2h eval
bb-connor a51808d
feat(cli): add --action-json/--action-file escape hatch to h2h eval
bb-connor c559d69
feat(cli): add --explain trace rendering and h2h explain subcommand
bb-connor b69d5d3
fix(cli): correct explain precedence notes and trim trace whitespace
bb-connor b48f7b9
feat(cli): add json and receipt output formats to h2h eval
bb-connor 88175ee
docs(cli): document h2h eval and h2h explain
bb-connor 0a3802c
docs(cli): use builtin:ai-agent in shell-command eval example
bb-connor 325e1d9
fix(cli): fail closed if eval output serialization ever fails
bb-connor c5d36c8
feat(receipt): add optional enforcement summary to receipt schema and…
bb-connor 7e5e469
feat(go): add enforcement summary to decision receipt
bb-connor 20a45c3
feat(ts): add enforcement summary to receipts and tagged observer events
bb-connor 6667975
feat(ts): add HushGuard enforcement config with validated per-rule ov…
bb-connor 9376fea
feat(ts): add HushGuard gate() with monitor mode and panic supremacy
bb-connor fec91ab
feat(ts): emit tagged receipts and observer events from HushGuard gate
bb-connor 762b8e6
feat(python): add enforcement summary to receipts and tagged observer…
bb-connor 78b4cbd
feat(python): add HushGuard enforcement config with validated per-rul…
bb-connor 302b231
feat(python): add HushGuard gate() with monitor mode and panic supremacy
bb-connor c70a440
feat(python): emit tagged receipts and observer events from HushGuard…
bb-connor a193eb4
docs: document monitor enforcement mode in core spec and SDK readmes
bb-connor 347cfeb
fix(middleware): validate extension segment in enforcement override keys
bb-connor 9efedcb
feat(cli): h2h lint --fix with decision-neutral auto-fixes
bb-connor 2a82765
ci: five-target release binaries with checksums and provenance
bb-connor 6381905
fix(ci): working release pipeline (runner image, ensure-release, tag …
bb-connor 03000af
ci: automated Homebrew tap formula updates on release
bb-connor 0a86765
fix(ci): fail-closed formula rendering and tap-push auth
bb-connor 1dbf17a
feat(dist): @hushspec/cli npm wrapper with per-platform binaries
bb-connor 2b65a04
fix(dist): portable npm shim entry-point detection
bb-connor f950ea9
docs: installation matrix for brew, npm, and prebuilt binaries
bb-connor f23e759
docs(cli): correct lint preview command and caveat wording
bb-connor 962c36c
test(cli): exercise a real fix in the neutrality corpus guard
bb-connor 55bfa03
test(schemas): meta-validation and $id guard for published schemas
bb-connor 2278ec5
fix(tests): assert schema formats explicitly under draft 2020-12
bb-connor 34f1eed
feat(fmt): preserve leading yaml-language-server modelines
bb-connor f9d8b43
fix(fmt): report parse errors against original line numbers
bb-connor 444a11e
feat: schema modelines on shipped policies and init scaffolds
bb-connor b61ffb8
feat(schemas): serve schemas at their $id URLs; SchemaStore entry and…
bb-connor 922e4b6
feat(testkit): add portable differential case-bundle format
bb-connor b5f5a94
test(fixtures): add input_inject evaluator fixture and widen action-t…
bb-connor d77c779
feat(testkit): add case verdicts and in-process Rust oracle
bb-connor a5f1d73
feat(testkit): add cross-SDK report comparison with divergence kinds
bb-connor f4fc5da
feat(testkit): add proptest policy/action generators and bundle gener…
bb-connor 2cc74d7
feat(testkit): add hushspec-gen bundle generator binary
bb-connor 3cbc0e0
feat(testkit): add fail-closed subprocess harness adapter
bb-connor 46efe87
test(testkit): cover subprocess fail-closed paths
bb-connor 68e6a17
feat(testkit): add TypeScript differential evaluation harness
bb-connor f453fc4
fix(evaluate): first-match-wins origin profile tie-break (cross-SDK p…
bb-connor f1e23e2
feat(testkit): add Python differential evaluation harness
bb-connor a0315f0
feat(testkit): add Go differential evaluation harness
bb-connor a21a66d
feat(testkit): add batch-shrinking divergence minimizer
bb-connor 4b5a285
feat(testkit): emit minimized divergences as evaluator regression fix…
bb-connor 8e1a2aa
fix(testkit): fail closed when a divergence can't be a valid fixture
bb-connor e1c81ba
feat(testkit): add hushspec-difftest differential fuzz runner
bb-connor 7dceb08
fix(testkit): guard minimizer against phantom/out-of-range divergence…
bb-connor a871506
feat(bench): add criterion benchmarks for evaluate and receipt overhead
bb-connor fd6172d
test(bench): gate receipt overhead against roadmap R10 budgets
bb-connor fbf60af
ci: add differential-fuzz and bench-thresholds jobs
bb-connor abd64f7
ci: add nightly deep-fuzz workflow and testkit docs
bb-connor eba557b
chore(tests): clean up clippy lints under --all-targets
bb-connor f58f0b5
fix(ci): drop --locked from release builds (no committed lockfile)
bb-connor 74ccc7e
fix(lint): collapse if into match guard (clippy 1.97 collapsible_if)
bb-connor 94131d3
chore: remove process/meta-narrative from code comments
bb-connor fd0beac
fix(middleware): audit the provider-failure would-block under monitor
bb-connor 015f537
fix: harden review findings across testkit, lint, and release scripts
bb-connor 85782eb
build: commit Cargo.lock and build release binaries --locked
bb-connor 82cb4c1
fix(cli): require --target for target-based eval action types
bb-connor 4bf2bc4
fix: resolve cross-SDK evaluation divergences and fail-open paths
bb-connor 6ca815c
fix(policy): broaden shipped secret detection and close credential-pa…
bb-connor 8c652b2
fix(receipt): keep the zero-overhead disabled-audit path; run cargo fmt
bb-connor ec1f8b8
fix(receipt): make content_hash optional so the disabled-audit path i…
bb-connor 674e987
fix(regex): reject catastrophic-backtracking patterns in the safety c…
bb-connor 4ac1e04
style: cargo fmt (validate.rs, receipt.rs)
bb-connor 66eb962
feat(detection): wire the content-detection engine into evaluation
bb-connor b10accf
fix(policy): close shipped-policy holes and make regexes engine-agnostic
bb-connor 6289f59
fix: cross-SDK validation and detector regex parity
bb-connor 9dd3061
fix: honor panic --sentinel in eval/test/diff and audit sink-only mon…
bb-connor 39ef8c5
fix: close cross-SDK parity divergences found in the parity audit
bb-connor 5149f3f
feat(sdk): resolve `extends: builtin:<name>` in the Python and Go SDKs
bb-connor b2071e5
fix(panic): include input_injection in the TS/Python emergency policy
bb-connor 2f8c73d
fix(policy): harden ai-agent and recommended shell_commands
bb-connor fc25664
fix: close remaining cross-SDK validation and conditions divergences
bb-connor 7fcb462
fix: close fail-open and robustness gaps from the fifth review
bb-connor 979ccf6
chore: harden builtin generators + publish workflow, widen fuzz coverage
bb-connor File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: Deploy Docs | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pages: write | ||
| id-token: write | ||
|
|
||
| concurrency: | ||
| group: pages | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| deploy: | ||
| environment: | ||
| name: github-pages | ||
| url: ${{ steps.deployment.outputs.page_url }} | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: peaceiris/actions-mdbook@v2 | ||
| with: | ||
| mdbook-version: latest | ||
|
|
||
| - name: Build book | ||
| run: mdbook build docs | ||
|
|
||
| - name: Publish schemas at their $id URLs | ||
| run: | | ||
| mkdir -p docs/book/schemas | ||
| cp schemas/*.json docs/book/schemas/ | ||
|
|
||
| - uses: actions/configure-pages@v5 | ||
|
|
||
| - uses: actions/upload-pages-artifact@v3 | ||
| with: | ||
| path: docs/book | ||
|
|
||
| - id: deployment | ||
| uses: actions/deploy-pages@v4 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| name: Nightly Fuzz | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '17 3 * * *' | ||
| workflow_dispatch: | ||
|
|
||
| env: | ||
| CARGO_TERM_COLOR: always | ||
|
|
||
| jobs: | ||
| differential-deep: | ||
| name: Differential Deep Fuzz | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: dtolnay/rust-toolchain@stable | ||
|
|
||
| - uses: Swatinem/rust-cache@v2 | ||
|
|
||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20 | ||
| cache: npm | ||
|
|
||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: '1.22' | ||
|
|
||
| - name: Install Node dependencies | ||
| run: npm ci | ||
|
|
||
| - name: Build TypeScript package | ||
| run: npm run build | ||
|
|
||
| - name: Install Python package | ||
| run: pip install -e "packages/python[dev]" | ||
|
|
||
| - name: Deep differential fuzz (random seed, minimized fixture candidates) | ||
| run: | | ||
| cargo run --release -p hushspec-testkit --bin hushspec-difftest -- \ | ||
| --chunks 100 --max-seconds 1500 \ | ||
| --groups 250 --actions-per-group 4 \ | ||
| --minimize \ | ||
| --emit-fixtures target/difftest/fixture-candidates \ | ||
| --report target/difftest/report.json | ||
|
|
||
| - name: Upload run artifacts (bundles, report, fixture candidates) | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: nightly-difftest | ||
| path: target/difftest/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| name: Release | ||
|
|
||
| on: | ||
| push: | ||
| tags: ["v*"] | ||
| workflow_dispatch: | ||
| inputs: | ||
| tag: | ||
| description: "Existing tag to (re)build artifacts for" | ||
| required: true | ||
|
|
||
| env: | ||
| TAG: ${{ github.event.inputs.tag || github.ref_name }} | ||
|
|
||
| jobs: | ||
| build: | ||
| strategy: | ||
| fail-fast: true | ||
| matrix: | ||
| include: | ||
| - { target: x86_64-unknown-linux-gnu, os: ubuntu-latest, cross: false } | ||
| - { target: aarch64-unknown-linux-gnu, os: ubuntu-latest, cross: true } | ||
| - { target: x86_64-apple-darwin, os: macos-15-intel, cross: false } | ||
| - { target: aarch64-apple-darwin, os: macos-14, cross: false } | ||
| - { target: x86_64-pc-windows-msvc, os: windows-latest, cross: false } | ||
| runs-on: ${{ matrix.os }} | ||
| steps: | ||
| - name: Validate tag format | ||
| shell: bash | ||
| run: | | ||
| [[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9.]+)?$ ]] || { echo "::error::TAG must look like v<semver>: $TAG"; exit 1; } | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| ref: ${{ env.TAG }} | ||
| - uses: dtolnay/rust-toolchain@stable | ||
| with: | ||
| targets: ${{ matrix.target }} | ||
| - uses: Swatinem/rust-cache@v2 | ||
| - name: Install cross | ||
| if: matrix.cross | ||
| run: cargo install cross --locked | ||
| - name: Build | ||
| shell: bash | ||
| run: | | ||
| if [ "${{ matrix.cross }}" = "true" ]; then | ||
| cross build -p hushspec-cli --release --locked --target ${{ matrix.target }} | ||
| else | ||
| cargo build -p hushspec-cli --release --locked --target ${{ matrix.target }} | ||
| fi | ||
| - name: Package | ||
| shell: bash | ||
| run: | | ||
| bin="h2h"; [ "${{ runner.os }}" = "Windows" ] && bin="h2h.exe" | ||
| stage="h2h-${TAG}-${{ matrix.target }}" | ||
| mkdir "$stage" | ||
| cp "target/${{ matrix.target }}/release/${bin}" LICENSE README.md "$stage/" | ||
| tar -czf "${stage}.tar.gz" "$stage" | ||
| - uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: h2h-${{ matrix.target }} | ||
| path: h2h-*.tar.gz | ||
| retention-days: 3 | ||
|
|
||
| release: | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
| id-token: write | ||
| attestations: write | ||
| steps: | ||
| - uses: actions/download-artifact@v4 | ||
| with: | ||
| merge-multiple: true | ||
| - name: Checksums | ||
| run: sha256sum h2h-*.tar.gz > SHA256SUMS | ||
| - uses: actions/attest-build-provenance@v2 | ||
| with: | ||
| subject-path: "h2h-*.tar.gz" | ||
| - name: Ensure release exists | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| gh release view "$TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1 || \ | ||
| gh release create "$TAG" --repo "$GITHUB_REPOSITORY" --verify-tag --generate-notes | ||
| - name: Upload to release | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: gh release upload "$TAG" h2h-*.tar.gz SHA256SUMS --repo "$GITHUB_REPOSITORY" --clobber | ||
|
|
||
| homebrew: | ||
| needs: release | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/download-artifact@v4 | ||
| with: | ||
| merge-multiple: true | ||
| - name: Regenerate checksums locally | ||
| run: sha256sum h2h-*.tar.gz > SHA256SUMS | ||
| - name: Render and PR formula | ||
| env: | ||
| GH_TOKEN: ${{ secrets.TAP_GITHUB_TOKEN }} | ||
| run: | | ||
| bash scripts/render_formula.sh "$TAG" SHA256SUMS > h2h.rb | ||
| gh repo clone backbay-labs/homebrew-tap tap | ||
| gh auth setup-git | ||
| cd tap | ||
| git checkout -b "h2h-${TAG}" | ||
| mkdir -p Formula && cp ../h2h.rb Formula/h2h.rb | ||
| git add Formula/h2h.rb | ||
| git -c user.name=hushspec-release -c user.email=noreply@backbay-labs.dev \ | ||
| commit -m "h2h ${TAG}" | ||
| git push origin "h2h-${TAG}" | ||
| gh pr create --repo backbay-labs/homebrew-tap \ | ||
| --title "h2h ${TAG}" --body "Automated formula update for ${TAG}." | ||
|
|
||
| npm-cli: | ||
| needs: release | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/download-artifact@v4 | ||
| with: | ||
| merge-multiple: true | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20 | ||
| registry-url: "https://registry.npmjs.org" | ||
| - name: Generate packages | ||
| run: node scripts/gen_npm_cli.mjs "$TAG" . out/ | ||
| - name: Publish (platform packages, then meta) | ||
| env: | ||
| NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} | ||
| run: | | ||
| for d in out/cli-*; do (cd "$d" && npm publish --access public --provenance); done | ||
| (cd out/cli && npm publish --access public --provenance) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.