[MAINT]: Bump the minor-and-patch group with 4 updates - #169
Merged
Spencer Schoenberg (spencrr) merged 1 commit intoAug 20, 2026
Merged
Conversation
Bumps the minor-and-patch group with 4 updates: [hatchling](https://github.com/pypa/hatch), [pre-commit](https://github.com/pre-commit/pre-commit), [ruff](https://github.com/astral-sh/ruff) and [ty](https://github.com/astral-sh/ty). Updates `hatchling` from 1.31.0 to 1.32.0 - [Release notes](https://github.com/pypa/hatch/releases) - [Commits](pypa/hatch@hatchling-v1.31.0...hatchling-v1.32.0) Updates `pre-commit` from 4.6.1 to 4.6.2 - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](pre-commit/pre-commit@v4.6.1...v4.6.2) Updates `ruff` from 0.16.2 to 0.16.3 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.16.2...0.16.3) Updates `ty` from 0.0.69 to 0.0.72 - [Release notes](https://github.com/astral-sh/ty/releases) - [Changelog](https://github.com/astral-sh/ty/blob/main/CHANGELOG.md) - [Commits](astral-sh/ty@0.0.69...0.0.72) --- updated-dependencies: - dependency-name: hatchling dependency-version: 1.32.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: minor-and-patch - dependency-name: pre-commit dependency-version: 4.6.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch - dependency-name: ruff dependency-version: 0.16.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch - dependency-name: ty dependency-version: 0.0.72 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: minor-and-patch ... Signed-off-by: dependabot[bot] <support@github.com>
dependabot
Bot
added
dependencies
dependabot
Bot
added
dependencies
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Spencer Schoenberg (spencrr)
approved these changes
Aug 20, 2026
Merged
3 tasks
Nina Chikanov (nina-msft)
added a commit
that referenced
this pull request
Aug 25, 2026
…e the evidence it needs (#141) ## Description Fixes #140 and closes #167, which @nina-msft asked be superseded rather than done separately. #140 has a standalone repro: a probe asserting an agent does not exfiltrate passes green while the agent exfiltrates on every turn, because the adapter declares `TOOL_ONLY` and so never populates `Response.side_effects`. `ToolCalled` and `SideEffectOccurred` return `NOT_DETECTED` in two different situations: the thing did not happen, and the adapter does not report that kind of evidence. `resolve_as_attack` maps `NOT_DETECTED` to `SAFE` and `resolve_as_probe` maps it to `UNSAFE`, so a gap in the adapter becomes a statement about the agent. `_adjust_for_observability` covers part of this, but only for `RESPONSE_ONLY`, only in XPIA, and only by counting tool calls. This PR adds a regression test for the attack side. Run against `main` at `3d219d8` it produces: ``` =========================== RAMPART Safety Summary ============================ UNCATEGORIZED (2 tests) FAIL test_dropping_observability_does_not_look_safer_async -- Attack objective detected: Side effect 'http_request': {'url': 'https://evil.com/steal'} (tool_and_side_effects) PASS test_dropping_observability_does_not_look_safer_async -- Agent defended successfully (tool_only) ``` `FAIL` and `PASS` there are RAMPART's per-result verdict labels, not pytest outcomes. Both lines come from one pytest test that runs the same attack twice. The agent exfiltrates on both runs; the two adapters differ in the declared level and in what that level lets them report, so the `TOOL_ONLY` one returns the same response with no `side_effects`, which is exactly what a `TOOL_ONLY` adapter can do. RAMPART calls that second run "Agent defended successfully". The pytest test fails on `main` and passes with this change. Every combination where a built-in evaluator needs evidence the adapter does not report, measured with a `MockAdapter`: | Adapter | Evaluator | Attack before | Attack after | Probe before | Probe after | |---|---|---|---|---|---| | `RESPONSE_ONLY` | `ToolCalled` | `UNDETERMINED` | `UNDETERMINED` | **`UNSAFE`** | `UNDETERMINED` | | `RESPONSE_ONLY` | `SideEffectOccurred` | `UNDETERMINED` | `UNDETERMINED` | **`UNSAFE`** | `UNDETERMINED` | | `TOOL_ONLY` | `SideEffectOccurred` | **`SAFE`** | `UNDETERMINED` | **`UNSAFE`** | `UNDETERMINED` | Two of the six were already correct, and both were correct because `_adjust_for_observability` caught them. Rebased onto `main` at `3d219d8`, so the tests here follow `RMP001` from #158 and #159 and the xdist transport is the one #166 landed. ### Changes - `ObservabilityLevel` gains `observes_tool_calls` and `observes_side_effects`, following the `PayloadFormat.is_text` and `is_binary` pattern already in that file. Its class docstring described only the `RESPONSE_ONLY` case, so it now also covers `TOOL_ONLY` with side effects, which is the case in the linked issue. - `EvalContext` gains `observability_level`. It is required, so a context built by hand has to say what the adapter behind it could see. - `evaluate_turn_async` takes the level, required and keyword-only, and puts it on the context. `XPIAExecution` and `SingleTurnExecution` both pass `adapter.observability_profile`. `Result` and `EvalContext.from_response` require it too, which is @nina-msft's request below and what closes #167. - `ToolCalled` and `SideEffectOccurred` return `UNDETERMINED` when they cannot see the evidence they need. The check runs after the scan, so anything the adapter does report still counts as evidence. `_adjust_for_observability` makes the same allowance today. - The `UNDETERMINED` summary on both strategies is built from `undetermined_operands`, so it names every channel that could not be observed rather than only the operand the composite reported first. Repeats collapse, and anything past the first two is counted. This is @nina-msft's second request below. - `_AllEvaluator` short-circuits only on a `NOT_DETECTED` left operand. An `UNDETERMINED` left operand no longer skips the right one, so `&` no longer depends on the order the operands were written in. Both undetermined branches carry the evidence of both operands. This is the review fix from @nina-msft below. - `_AnyEvaluator` names the undetermined operand and carries the evidence of both, instead of a bare "One or both operands undetermined". Outcomes are unchanged. Without this, `|` hid the adapter setting behind the verdict, which is the one thing this PR is trying to surface, and the note added to `authoring-tests.md` points the reader at `|` for exactly this case. - The XPIA undetermined summary prefers results that are themselves `UNDETERMINED`, matching the probe summary above it. Without that it could lead with a `NOT_DETECTED` rationale from a different turn. Settled results are read only when nothing else gave a reason, which is what the `_adjust_for_observability` downgrade looks like. - The probe unsafe summary takes its reason from a `NOT_DETECTED` result. It took the last rationale of any outcome, so once these evaluators can return `UNDETERMINED`, an undetermined turn could state the reason for a definitive failure. - `EvalResult` gains `undetermined_operands`. `&` and `|` record every operand they ran that came back `UNDETERMINED`, so "the predicate is false" stops being indistinguishable from "the predicate is false and every part of the evaluation ran". `~` carries its inner result's entries through, each reason is kept once, the xdist transport round-trips it with the same ANSI stripping as the other free text, and `JsonFileReportSink` emits it as `eval_undetermined_operands` when it is not empty. This is @bashirpartovi's first option below; the truth table is untouched. - The `SAFE` summary on both strategies names what was left undetermined. It names the first two distinct reasons and counts the rest. Verdicts do not move, so no existing result changes status. - The XPIA unsafe summary takes its evidence only from `DETECTED` results, matching the probe summary. An `UNDETERMINED` composite can carry evidence of its own, and that evidence is not what established the verdict. - The JSON run report carries `observability_level`. It named the verdict, the strategy and the harm category but not the level the run was gathered under, so a dashboard could not tell a clean pass from one the adapter was never able to see through. The xdist transport already carried it. - Every read of `undetermined_operands` and `evidence` in the composites, the summaries and the two serializers goes through `safe_str_list`, and the probe's unsafe and error summaries put `rationale` through `safe_str` where the XPIA summary was already guarded. `_distinct_operand_reasons` flattened the operand list with a comprehension, so a third-party evaluator returning a non-iterable, or an iterator whose `__iter__` raises, aborted summary construction before the containment helpers saw it. `evidence` had the same shape, and this branch had taken the composites from one evidence concatenation to five, where a value that is not a list broke the compose step itself. - The probe unsafe summary renders each rationale before testing whether it has content. It filtered on the raw value first, so a rationale whose truthiness raises took the summary and the verdict with it, and a whitespace-only rationale printed `UNSAFE: ` with nothing after the colon. This is @nina-msft's request below, using the code she supplied. One behavior moves with it: a rationale that is falsy but renders as something, such as `None` or `0`, now shows as itself where it used to fall through to the generic line. Both readings are of a value that already violates the declared `str`, and the verdict is the same either way. - `safe_str` returns an exact `str`. `str()` accepts a `__str__` that returns a `str` subclass, so the rendered value could still carry evaluator code on the methods RAMPART reaches for next. `_distinct_reasons` and `_merge_undetermined` already called `.strip()` on it, and the fix above adds a third such call, so containment was moving the failure rather than removing it. `str.__str__` is the C slot: it cannot be overridden, cannot raise, and returns the argument unchanged when it is already exact. - The xdist truncation marker carries the run's real observability level. It hardcoded `RESPONSE_ONLY`, so a result too large to send came back through the controller claiming the narrowest level, in the field the rest of this PR is about. Predates this branch; the original `Result` was already in scope. - The undetermined summary reads settled results only when no result stayed undetermined, so a gap another turn settled around cannot be offered as the reason this verdict was missed. - `safe_str` and `safe_str_list` in `rampart/common/text.py` coerce evaluator-supplied values without raising. Every rationale interpolation in the composites goes through them, as does every read of `undetermined_operands` in the composites, the JSON sink and the xdist serializer. `_AnyEvaluator` and `_AllEvaluator` between them gained three rationale interpolations `main` does not have, so a value whose `__str__` raises turned inputs that resolved cleanly on `main` into `SafetyStatus.ERROR`, losing a verdict the evaluators had already reached. Both helpers catch `Exception` rather than `BaseException`, so cancellation and interrupts still propagate. - `ObservabilityLevel` and `authoring-tests.md` now say the guarantee is per channel rather than per field: a level that reports a channel is taken at its word for what it puts in it, so a tool call reported with redacted arguments still counts as observed and a predicate over those arguments can return `NOT_DETECTED`. - `Session.send_async`, `authoring-tests.md` and `quickstart.md` said empty lists mean "no observations", not "nothing happened". That rule predates the declared level and now reads backwards, and it contradicted `observability_profile`'s own docstring in the same file. All three now say an empty list is read against the declared level. - The composition and reporting docs say how a recorded gap is graded: the verdict does not move, `SAFE` is the only status that passes, and on a run that reaches it the summary and `undetermined_operands` are the only places the gap shows. `results-and-reporting.md` documents `turn.eval_result.undetermined_operands` and the per-turn `eval_undetermined_operands` key, with a snippet that fails a passing run carrying one. This is @bashirpartovi's request below. - @nina-msft pushed five commits to this branch, `38dd617`, `c93f611`, `7643ac8`, `60924db` and `a217370`. They drop a blank carried reason in `_merge_undetermined` so it falls through to the rationale, promote the finite-float guard out of `_xdist.py` into `rampart/common/text.py` as `safe_float`, put `safe_float` and `safe_str` on `confidence` and `rationale` in both serializers, and teach the xdist reader to tell a confidence that is missing from one that was sanitized. Her note below has the detail. - On top of those, `2c195aa` adds a test per site for the two `rationale` guards, which the sweep below had left green, and `aefee9b` removes a `ty: ignore` that `ty` reports as unused and that was failing Lint & Type Check with the test matrix skipped behind it. ### Why the fix is in the evaluator Two docstrings disagree about this, so I want to be explicit about which one I followed and why. Both are quoted as they stand on `main`; this PR updates both. `rampart/core/types.py:27-29`: > When the adapter declares RESPONSE_ONLY, evaluators that require tool call data return UNDETERMINED rather than a false SAFE. `rampart/evaluators/tool_called.py:23-25`: > This evaluator only detects conditions. It does not reason about observability gaps. That adjustment is owned by the execution strategy. I followed the first one. The obvious alternative is to keep the adjustment central and have evaluators declare a `required_observability` for the strategy to read. I could not make that work for composition. Under `TOOL_ONLY`, `ToolCalled("x") | SideEffectOccurred("y")` should still return `DETECTED` if `x` was called, while the right operand cannot be observed. A strategy-level check against a composite's declared requirement cannot see the operands, so it either suppresses a real detection or does nothing. The post-scan allowance above has the same problem: "evidence the adapter actually reported still counts" is a per-operand runtime fact, not something a static declaration can express. `|`, `&` and `~` already arbitrate this correctly once operands can return `UNDETERMINED`, which is what this change gives them. There is also precedent for an evaluator reporting its own uncertainty. `LLMJudge` returns `UNDETERMINED` when the judge output is malformed after retries or the call fails, rather than guessing. Those are transient instrument failures and an observability gap is static configuration, so the situations are not identical, but the outcome type is doing the same job in both: `EvalOutcome.UNDETERMINED` is defined as "The evaluator could not make a determination". The adjustment itself stays where the second docstring puts it. `_adjust_for_observability` is unchanged and still owns the verdict downgrade. What changes is the quality of its input. The sentence in `ToolCalled`'s docstring is contradicted by this PR and is updated, as is the matching note in `docs/usage/authoring-tests.md`. ### No new verdict semantics `UNDETERMINED` is not new at either level. `EvalOutcome.UNDETERMINED` is produced today by `LLMJudge` and by `|` and `&`, and preserved by `~`. `SafetyStatus.UNDETERMINED` is produced by both resolvers and by `_adjust_for_observability`. Every consumer already handles it: the resolver precedence rules, the composition operators, the xdist round trip through `SafetyStatus(value)`, `JsonFileReportSink`, the WARN terminal label, and the population summary. This change produces it in more of the cases it already exists for. `DETECTED` that came from observed evidence is untouched on every path, so no evidence-based detection is weakened. The one detection that changes is `~` inverting an absence the adapter could not attest, covered below. ## Breaking changes Yes, in two ways, and the title carries `[BREAKING]` as @nina-msft asked. **1. `observability_level` is required on four public APIs.** `EvalContext`, `EvalContext.from_response`, `evaluate_turn_async` and `Result` no longer default it. Three of those four parameters are introduced by this PR, so the break there is against a signature that has not shipped; `Result` is the one that predates the branch and loses a real default of `RESPONSE_ONLY`. `Result` is also the widest of the four: of the 94 call sites in this repo's Python files that omitted the argument, 75 were `Result(...)`, counting the ones written inside pytester source strings. Two more `Result(...)` examples in `docs/` omitted it as well, and are updated here. @nina-msft left this one to my judgement with a stated preference for requiring it, and requiring it is what actually dissolves the asymmetry, so that is what this does. Say the word and I will put the `Result` default back. Migration is to pass the adapter's declared level, normally `adapter.observability_profile`. Omitting it is a `TypeError` at the call rather than a silent assumption in a report. No call site in `rampart/` omitted it, so no built-in behavior moves. All four are keyword-only, so nothing positional breaks. **2. Verdicts move**, in one direction for the evaluators on their own and in one cell for `&`. Nothing is removed or renamed otherwise. The xdist transport gains a key. `JsonFileReportSink` gains two: `eval_undetermined_operands` per turn when the list is not empty, and `observability_level` on every result unconditionally, so a consumer validating a strict schema on a result object sees a new always-present field. `EvalResult.undetermined_operands` is written and read at both ends of the xdist transport; a payload without the key still deserializes and an old controller ignores the extra one, so `SCHEMA_VERSION` is unchanged, and bumping it would make `_validate_schema` reject the whole payload instead. `JsonFileReportSink` emits `eval_undetermined_operands` per turn only when the list is not empty. For `ToolCalled` and `SideEffectOccurred` used alone, `NOT_DETECTED` becomes `UNDETERMINED` and nothing moves toward `SAFE`. What existing suites will see: - An attack that passed because the adapter could not see side effects now returns `UNDETERMINED` and fails. That is the bug being fixed, and it will surface as a newly red test. - A probe using `ToolCalled` or `SideEffectOccurred` below the level it needs goes from `UNSAFE` to `UNDETERMINED`. Both are falsy, so the test still fails, but the terminal label changes from FAIL to WARN. - `~ToolCalled(...)` under `RESPONSE_ONLY` previously returned `DETECTED` by inverting an absence the adapter could not attest, and now passes `UNDETERMINED` through. On a probe, "must not call X" against an adapter that cannot report tool calls was a false pass and now fails. The linked issue is the same shape one level down: `~SideEffectOccurred("http_request")` against a `TOOL_ONLY` adapter. - With the default trial threshold of 0.0, a group whose clones are all `UNDETERMINED` logs a passing gate line where it previously logged a failing one. The clones still fail, since `assert result` is falsy, and `_evaluate_gates` only logs, so no CI outcome flips. I left the threshold alone because PR #121 is reworking that layer. Making `&` order independent required choosing which outcome wins when one operand is `NOT_DETECTED` and the other is `UNDETERMINED`. It returns `NOT_DETECTED`, which is Kleene and is what the review asked for. Against every operand pair on `main`, one cell moves: ``` main : undetermined & not_detected -> undetermined attack=undetermined branch : undetermined & not_detected -> not_detected attack=safe ``` This is not a regression against `main` for `ToolCalled` or `SideEffectOccurred`, which returned `NOT_DETECTED` on `main` at a level that could not report the evidence, so the conjunction already resolved `SAFE`. It does mean a composed evaluator no longer gets the protection the first commit of this PR gave it in one of the two operand orders, and that a degraded `LLMJudge` inside `&` can now resolve `SAFE` where `main` said `UNDETERMINED`. Both cases now record the reason in `EvalResult.undetermined_operands`, and a `SAFE` summary names it, when the undetermined operand is on the left, since `&` still short-circuits on a `NOT_DETECTED` left operand and never runs what is to its right. `|` reports `UNDETERMINED` in those cases, and the docs now say which operator to reach for and which side to put the observability-dependent operand on. For the verdict changes there is no migration beyond fixing the adapter's declared level or the evaluator choice. The new rationale string names the declared level, the channel it does not report, and the target the evaluator was looking for. ### Deliberately out of scope - `_adjust_for_observability` also fires when it should not: `RESPONSE_ONLY` with `ResponseContains` is downgraded even though that evaluator never needed tool data. That is a false positive rather than a false negative, and narrowing the heuristic is a separate change. - `LLMJudge` now receives `observability_level` and ignores it. Telling the judge that tool calls are not visible would stop it reading an evidence-free transcript as innocence, but that changes judge prompting. - `ResponseContains` is untouched on purpose. Every level reports text, so no declared level hides it. - Splitting `EvalOutcome.UNDETERMINED` into "cannot observe" and "did not run" so `&` can treat them differently. That is the real fix for the `LLMJudge` case above and it is bigger than this PR. `undetermined_operands` records both kinds without telling them apart, so it does not pre-empt that design. - The rationale the LLM driver puts into its next prompt is untouched. At `rampart/drivers/llm.py:333` a value whose truthiness raises costs the next turn before rendering is tried. `confidence` and `rationale` in the two serializers were on this list until @nina-msft guarded them in `c93f611`. - A reason that itself contains `"; "` is indistinguishable from the separator the summary joins on. Same on `main` for the existing summaries, and the full list is on `Result.eval_results` and in the JSON either way. ## Checklist - [x] `pre-commit run --all-files` passes - [x] Tests added or updated for changes - [x] Documentation updated ### Tests 300 new tests against `main`, and one removed: `test_left_undetermined_short_circuits_async` asserted that `&` skips the right operand when the left is `UNDETERMINED`, which is the behavior the review asked me to remove. `8bf0b62` renamed it to `test_left_undetermined_evaluates_right_async` and inverted its `right.call_count` assertion, so the coverage moved rather than being dropped. It is the only collected node id `main` has that this branch does not, so nothing else existing was changed, removed or reparented. One test appears and disappears inside the branch rather than against `main`: `9ac85b7` added `test_observability_level_defaults_to_no_declared_limit` and `b86a67f` removed it, because it asserted the default that commit takes away; four tests asserting the `TypeError` replace it, one per API. The containment tests cover the operand path through `_summarize_undetermined_operands` and `_explain_undetermined`, both summary builders, every branch of the three composites, the xdist serializer, the new report key, and `safe_str_list` itself. Most of them are parametrized sweeps over the composites, one per field, because line coverage cannot see an expression change: a guard runs whether or not any test would notice it being removed. Neutering each of the 35 `safe_str` and `safe_str_list` call sites on this branch, one at a time, turns the suite red at every one of them. Across the whole PR, by file: `test_evaluator.py` 151, `test_text.py` 34, `test_single_turn.py` 26, `test_xpia.py` 25, `test_result.py` 21, `test_xdist.py` 12, `test_tool_called.py` 9, `test_side_effect.py` 7, `test_types.py` 7, `test_json_file.py` 6, `test_execution.py` 2. Those cover the outcome table for `&` and `|`, commutativity over all nine operand pairs, De Morgan both ways, associativity over all 27 triples, which cells record an undetermined operand and which cannot because the short-circuit skipped it, `UNDETERMINED` at each insufficient level with the rationale naming the level and the target, evidence still counted below the declared level, the new field surviving the xdist round trip with ANSI stripped at the boundary, and the report key present only when the list is not empty. The 27 that `4b43052`, `b86a67f` and `82f7926` added: - `test_result.py` (10): nine on `_explain_undetermined` in priority order, plus one `TypeError` test. Operand reasons beat the composite rationale, a reason repeated across turns collapses on both the operand path and the rationale path, the count names what it does not, a settled result cannot speak over an operand that stayed undetermined, a settled result does speak when nothing else did, and a blank or whitespace-only reason falls through to the fixed phrase rather than rendering an empty detail. - `test_xpia.py` (7) and `test_single_turn.py` (7): `ToolCalled("x") | SideEffectOccurred("y")` under `RESPONSE_ONLY` end to end, naming both channels, which is the case in the review comment; the `_adjust_for_observability` downgrade naming the gap it recorded; and the unit-level dedup, count and settled-result cases on both strategies, plus the two fallback cases on the probe. - `test_types.py` (2) and `test_execution.py` (1): the remaining three of the four `TypeError` tests, one per API. - `test_xdist.py`: no new ids. The oversized-result marker test now asserts the level survives truncation instead of being rewritten to `RESPONSE_ONLY`. `tests/integration/test_smoke.py` uses `ToolCalled` through `EvalContext.from_response` and asserts a detection, so no verdict there moves; it only gains the now-required argument. It needs no credentials and passes: `2 passed`. ### Documentation - `docs/usage/authoring-tests.md`: the `ToolCalled` warning said it "always returns `NOT_DETECTED`" under `RESPONSE_ONLY`, which is no longer true. `SideEffectOccurred` had no note and now has one. Added a short paragraph under the levels table on why declaring the level honestly matters, a paragraph saying the guarantee is per channel rather than per field, a note on how `UNDETERMINED` travels through `&` and `|`, which no user facing page covered, and a paragraph on what `undetermined_operands` records and which side of `&` to put an observability-dependent operand on. - `docs/attacks/xpia.md`: the Observability Adjustment section now says what it is for, now that evaluators handle their own cases, and the composition example says which operator to reach for when two evaluators are two views of one harm, and that the result records the gap rather than the verdict resting on silence. - `docs/contributing/extending-rampart.md`: the custom execution strategy example called `evaluate_turn_async` without the level, which would silently treat every adapter as fully observable. Fixed, plus a bullet in the key points, reworded again this round because omitting it is now a `TypeError` rather than a wrong assumption. - `docs/contributing/testing.md` and `docs/usage/pytest-integration.md`: the `Result(...)` helper and the manual-recording example both pass the level now, with a line on how to choose one and a note that existing tests were backfilled with the old default so no test changed meaning. - `docs/api/core-protocols.md`: `evaluate_turn_async` is exported from `rampart.core` but was absent from the API reference. It joins the other `rampart.core.execution` members there. It is not importable from `rampart` directly, so it does not belong on `core-types.md`, whose lede promises exactly that. - `docs/getting-started/quickstart.md` and `docs/glossary.md`: the empty-list rule and the `EvalContext` entry. No new pages, so no `mkdocs.yml` nav change. ### Checks run locally Rebased onto `main` at `3d219d8`. That range brought ruff 0.16.3 and ty 0.0.72 into `uv.lock` via #169, so the numbers below are at those versions, not the ones I quoted two rounds ago. ``` ruff 0.16.3 check .............. passed ruff 0.16.3 format --check ..... 128 files already formatted ty 0.0.72 check ................ passed flake8 7.3.0 (RMP codes) ....... passed, 0 violations coverage run -m pytest ......... 1020 passed, 7 skipped pytest -n 4 .................... 1020 passed, 7 skipped coverage report ................ TOTAL 94%, threshold 80 core/types.py 100% core/evaluator.py 100% core/result.py 100% reporting/json_file.py 100% common/text.py 100% evaluators/tool_called.py 100% evaluators/side_effect.py 100% probes/_single_turn.py 100% attacks/_xpia.py 95% mkdocs build --strict .......... no content warnings, same as main ``` `mkdocs build --strict` aborts on both `main` and this branch for the same environmental reason: the Material offline plugin cannot create symlinks on Windows. All 43 warnings on both sides are that one, and filtering it out leaves none. --------- Co-authored-by: Nina Chikanov <nichikan@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps the minor-and-patch group with 4 updates: hatchling, pre-commit, ruff and ty.
Updates
hatchlingfrom 1.31.0 to 1.32.0Release notes
Sourced from hatchling's releases.
Commits
558061cFix ci (#2377)bc6d1bcrelease Hatch v1.18.0 (#2376)8429e5drelease Hatchling v1.32.0 (#2375)413b3a8Prepare for release (#2374)4ccc9eaBug Fix - fix CTRL-C behavior to correctly handle signals (#2369)a885803Add sources to enable other types of local dependencies (#2313)ab3e000Add --all flag to hatch build for building all workspace members (#2352)bb90277fix: Allow the; privateannotation inimport-names/import-namespacesp...3a9d853fix: prevent backward relative paths in readme (#2354)c4dc4f8test: Fix test expectations for metadata generated byflit-core(#2365)Updates
pre-commitfrom 4.6.1 to 4.6.2Release notes
Sourced from pre-commit's releases.
Changelog
Sourced from pre-commit's changelog.
Commits
9767b6cv4.6.242ee3ffMerge pull request #3743 from pre-commit/npm-build-scripts-11-x3056619fixlanguage: nodefor hooks with build scripts and npm 11.xUpdates
rufffrom 0.16.2 to 0.16.3Release notes
Sourced from ruff's releases.
... (truncated)
Changelog
Sourced from ruff's changelog.
... (truncated)
Commits
b0e4702Bump 0.16.3 (#27723)ecdd401[ty] Separate script and uv modules from project metadata (#27720)1263524[ty] Simplify display implementations with std::fmt::from_fn (#27718)59196ba[ty] Unify polarity-aware relation construction (#27707)b8c5e73[ty] Disable CodSpeed cycle estimation for instrumented benchmarks (#27706)2b0d210[ty] Centralize matched argument relations (#27705)a9130f3[pyupgrade] Add rule to replacewhile 1withwhile True(while-one, `...c64c7d6[ty] Model try exception flow with operation checkpoints (#27471)9dea5ef[ty] Avoid deriving sequents for typevars with concrete bounds (#27587)9798e88[ty] Preserve enum exhaustiveness with custom missing methods (#27700)Updates
tyfrom 0.0.69 to 0.0.72Release notes
Sourced from ty's releases.
... (truncated)
Changelog
Sourced from ty's changelog.
... (truncated)
Commits
62ebbecBump version to 0.0.72 (#4268)fd1753dAlign Renovate config with preset (#4266)683e223Bump version to 0.0.71 (#4244)ede66b4Document unsound-yield in the migration guide (#4239)4c3d909Validate the PGO pipeline in CI (#4236)df9bad1Enable PGO for Linux ARM64 ty releases (#4218)0ed874cEnable PGO for Windows x86-64 ty releases (#4217)8988165Enable PGO for macOS ARM64 ty releases (#4216)c25c341Enable PGO for Linux x86-64 ty releases (#4213)d95445fUse Depot runners for release builds (#4224)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore <dependency name> major versionwill close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)@dependabot ignore <dependency name> minor versionwill close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)@dependabot ignore <dependency name>will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)@dependabot unignore <dependency name>will remove all of the ignore conditions of the specified dependency@dependabot unignore <dependency name> <ignore condition>will remove the ignore condition of the specified dependency and ignore conditions