From a64111407b964ce08166084e82749192c3e78611 Mon Sep 17 00:00:00 2001 From: binaryknight Date: Tue, 10 Nov 2026 23:31:52 +0000 Subject: [PATCH] Release 1.0.0 Set the version to 1.0.0 and group the changelog for this release, the first covered by docs/COMPATIBILITY.md. The Development Status classifier becomes Production/Stable, the README's Action example pins @v1.0.0, and the roadmap's Done list gains a 1.0.0 entry. The license is now declared as the SPDX expression MIT with license-files, which needs setuptools 77 or later. The table form it replaces is deprecated and stops building in 2027, and setuptools does not accept a license classifier beside an expression, so that classifier is removed. The schemas say agentsec_version is absent from reports written before 1.0.0, the first release that records it. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 37 ++++++++++++------- README.md | 2 +- agentsec/__init__.py | 2 +- agentsec/data/blast-radius-report.schema.json | 2 +- agentsec/data/path-assertions.schema.json | 2 +- docs/ROADMAP.md | 5 ++- pyproject.toml | 10 ++--- 7 files changed, 36 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7171ad9..cf393a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,19 +4,30 @@ What changed in each release of `agentsec-pack`, newest first. Probe versions are listed separately because a report records the probe version that took it, and two reports are only comparable when it matches or the difference is known. -## Unreleased - -- Under a user-space kernel such as gVisor, findings that describe what that kernel grants the process (PRIV-001 to PRIV-005, FS-005, FS-006 and RES-001) are reported at info, with their usual severity kept in the evidence, because that kernel is the sandbox's boundary and not the host's. Findings about what the sandbox exposes beyond it keep their severity. The rule does not apply when something is mounted over `/proc` or `/proc/version`. This is an exception, made before 1.0, to the compatibility policy's rule that an existing finding's severity changes only in a new major version. -- Modal's four reports were rescored from the same probe output and now read 46, 39, 39 and 31, where they read 100, 100, 100 and 97. A Modal baseline taken before this change has no `agentsec_version`, so `--baseline` shows the drop without a version note. Nothing fails, since a drop is not a regression. -- The version on `main` is 0.5.0.dev0, so a report written from it says which scorer produced it. -- `docs/COMPATIBILITY.md` states what stays the same from one version to the next: the commands, flags and exit codes, the report format, the scoring and the GitHub Action's inputs and outputs. `tests/test_compatibility.py` checks each of those. `verify`, `submit` and `benchmark` are marked experimental in their help text. -- Reports now record `agentsec_version`, the version that wrote and scored them, beside the probe's `probe_version`. `--baseline` and `agentsec history` note when it differs, as they already did for the probe. JSON Schemas for the blast-radius report and the `assert` result ship in `agentsec/data/`. -- `agentsec --version` prints the package and probe versions. -- The README said a probe version difference fails `--baseline`. It fails only when a finding is new or more severe, which is what the gate has always done, and a version difference with nothing worse passes with a note. The README and the note now say so. -- The README's GitHub Action example pins `@v0.4.0`, the latest release, instead of `@v0.2.0`. -- The README opens with a 63-second explainer video: what the probe measures, the `python:3.12-slim` hardening scores, the Antigravity CLI 1.5.2 and `srt` 0.0.81 measurements, and the Harbor grader control task, with each figure taken from a report in `examples/`. -- Antigravity CLI 1.5.2 measured, with writes and proxy traffic tested for real by `scripts/check_sandbox_boundaries.py`. Under `--sandbox`, `--dangerously-skip-permissions` now leaves the home directory, its shell startup files and the project writable and scores 100, and the 1.5.1 proxy still accepts tunnels to any hostname. With `proceed-in-sandbox` the score stays 31 and the working directory stays read-only, while files written to the build caches it leaves writable, including `~/.cargo/bin`, persist on the host. `measure_stubbed_agent.py` gains `--script` to run a check in place of the probe. -- Anthropic's sandbox runtime, `srt` 0.0.81, measured on macOS in four configurations. With `allowWrite ["."]` it leaves Claude Code's project settings files writable, which its documentation describes, and naming them in `denyWrite` protects them. +## 1.0.0 — 2026-11-10 + +Probe **0.7.0**, unchanged since 0.4.0. The first release covered by `docs/COMPATIBILITY.md`: from here on, a change that breaks one of its promises waits for 2.0. + +**Compatibility** +- `docs/COMPATIBILITY.md` states what stays the same from one version to the next: the commands, flags and exit codes, the report format, the scoring and the GitHub Action's inputs and outputs. `tests/test_compatibility.py` checks each of those. `verify`, `submit` and `benchmark` are marked experimental in their help text. (#51) +- Reports now record `agentsec_version`, the version that wrote and scored them, beside the probe's `probe_version`. `--baseline` and `agentsec history` note when it differs, as they already did for the probe. JSON Schemas for the blast-radius report and the `assert` result ship in `agentsec/data/`. (#51) +- `agentsec --version` prints the package and probe versions. (#51) +- The README said a probe version difference fails `--baseline`. It fails only when a finding is new or more severe, which is what the gate has always done, and a version difference with nothing worse passes with a note. The README and the note now say so. (#51) + +**Scoring** +- Under a user-space kernel such as gVisor, findings that describe what that kernel grants the process (PRIV-001 to PRIV-005, FS-005, FS-006 and RES-001) are reported at info, with their usual severity kept in the evidence, because that kernel is the sandbox's boundary and not the host's. Findings about what the sandbox exposes beyond it keep their severity. The rule does not apply when something is mounted over `/proc` or `/proc/version`. This is an exception, made before 1.0, to the compatibility policy's rule that an existing finding's severity changes only in a new major version. (#53) +- Modal's four reports were rescored from the same probe output and now read 46, 39, 39 and 31, where they read 100, 100, 100 and 97. A Modal baseline taken before this change has no `agentsec_version`, so `--baseline` shows the drop without a version note. Nothing fails, since a drop is not a regression. The rescored reports name `0.5.0.dev0`, the development version that rescored them before this release. (#53) + +**Measured profiles** +- Antigravity CLI 1.5.2 measured, with writes and proxy traffic tested for real by `scripts/check_sandbox_boundaries.py`. Under `--sandbox`, `--dangerously-skip-permissions` now leaves the home directory, its shell startup files and the project writable and scores 100, and the 1.5.1 proxy still accepts tunnels to any hostname. With `proceed-in-sandbox` the score stays 31 and the working directory stays read-only, while files written to the build caches it leaves writable, including `~/.cargo/bin`, persist on the host. `measure_stubbed_agent.py` gains `--script` to run a check in place of the probe. (#45) +- Anthropic's sandbox runtime, `srt` 0.0.81, measured on macOS in four configurations. With `allowWrite ["."]` it leaves Claude Code's project settings files writable, which its documentation describes, and naming them in `denyWrite` protects them. (#46) + +**Packaging** +- PyPI classifiers, with Development Status 5 - Production/Stable. The license is declared as the SPDX expression `MIT`, which replaces the deprecated table form and needs setuptools 77 or later to build. (#51) + +**Documentation** +- The README opens with a 63-second explainer video: what the probe measures, the `python:3.12-slim` hardening scores, the Antigravity CLI 1.5.2 and `srt` 0.0.81 measurements, and the Harbor grader control task, with each figure taken from a report in `examples/`. (#48) +- The README's GitHub Action example pins the latest release, `@v1.0.0`, instead of `@v0.2.0`. (#49, #50) ## 0.4.0 — 2026-11-07 diff --git a/README.md b/README.md index 585c7c6..d27355a 100644 --- a/README.md +++ b/README.md @@ -171,7 +171,7 @@ Both `--baseline` and `agentsec diff` exit with code 2 and list each finding tha GitHub Actions runs the same gate checks in one workflow step by installing `agentsec` from the action's own checkout, so the version pinned in the workflow is the version that measures. ```yaml -- uses: binary-knight/agentsec-pack@v0.4.0 # a release tag, which sets the version of agentsec +- uses: binary-knight/agentsec-pack@v1.0.0 # a release tag, which sets the version of agentsec with: image: myagent:${{ github.sha }} preset: docker-hardened # or leave out and pass `flags:` instead diff --git a/agentsec/__init__.py b/agentsec/__init__.py index 9aac781..eaa1435 100644 --- a/agentsec/__init__.py +++ b/agentsec/__init__.py @@ -1,2 +1,2 @@ """agentsec-pack: adversarial tests for AI agent deployments.""" -__version__ = "0.5.0.dev0" +__version__ = "1.0.0" diff --git a/agentsec/data/blast-radius-report.schema.json b/agentsec/data/blast-radius-report.schema.json index 4ffde21..196a6c7 100644 --- a/agentsec/data/blast-radius-report.schema.json +++ b/agentsec/data/blast-radius-report.schema.json @@ -9,7 +9,7 @@ "tool": {"const": "agentsec-pack"}, "test": {"const": "sandbox-blast-radius"}, "schema_version": {"const": 1}, - "agentsec_version": {"type": "string", "description": "The agentsec-pack version that wrote and scored the report. Absent from reports written before 0.5.0."}, + "agentsec_version": {"type": "string", "description": "The agentsec-pack version that wrote and scored the report. Absent from reports written before 1.0.0."}, "label": {"type": ["string", "null"]}, "generated_at": {"type": "integer", "description": "Unix time, seconds."}, "target": {"$ref": "#/$defs/target"}, diff --git a/agentsec/data/path-assertions.schema.json b/agentsec/data/path-assertions.schema.json index 61f657b..8193e5d 100644 --- a/agentsec/data/path-assertions.schema.json +++ b/agentsec/data/path-assertions.schema.json @@ -9,7 +9,7 @@ "tool": {"const": "agentsec-pack"}, "test": {"const": "path-assertions"}, "schema_version": {"const": 1}, - "agentsec_version": {"type": "string", "description": "The agentsec-pack version that wrote the result. Absent from results written before 0.5.0."}, + "agentsec_version": {"type": "string", "description": "The agentsec-pack version that wrote the result. Absent from results written before 1.0.0."}, "generated_at": {"type": "integer", "description": "Unix time, seconds."}, "target": { "type": "object", diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index a29961d..c21bbf9 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -57,9 +57,10 @@ The rules `agentsec benchmark` enforces: ## Done -**Unreleased, on main** -- Scoring under a user-space kernel: findings that describe what gVisor's kernel grants are reported at info, so Modal's scores count what the sandbox exposes beyond gVisor. +**1.0.0, 2026-11-10** - A written compatibility policy, with a test for each promise, and JSON Schemas for reports. +- Scoring under a user-space kernel: findings that describe what gVisor's kernel grants are reported at info, so Modal's scores count what the sandbox exposes beyond gVisor. +- Antigravity CLI 1.5.2 measured with writes and proxy traffic tested for real, and Anthropic's sandbox runtime measured on macOS. **0.4.0, 2026-11-07** - Probe 0.7.0, which no longer reports an allowlist that filters at TLS as open internet, and flags gVisor. diff --git a/pyproject.toml b/pyproject.toml index d6e1017..1430941 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,19 +1,19 @@ [build-system] -requires = ["setuptools>=68"] +requires = ["setuptools>=77"] build-backend = "setuptools.build_meta" [project] name = "agentsec-pack" -version = "0.5.0.dev0" +version = "1.0.0" description = "Measure what an AI agent's sandbox lets it reach: network, secrets, host filesystem, privileges and processes, from inside the sandbox." readme = "README.md" requires-python = ">=3.10" -license = {text = "MIT"} +license = "MIT" +license-files = ["LICENSE"] classifiers = [ - "Development Status :: 4 - Beta", + "Development Status :: 5 - Production/Stable", "Environment :: Console", "Intended Audience :: Developers", - "License :: OSI Approved :: MIT License", "Operating System :: MacOS", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 3 :: Only",