Skip to content

chore(deps-dev): bump pytest from 8.4.2 to 9.0.3 in /scripts/benchmarks - #2448

Open
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/uv/scripts/benchmarks/pytest-9.0.3
Open

chore(deps-dev): bump pytest from 8.4.2 to 9.0.3 in /scripts/benchmarks#2448
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/uv/scripts/benchmarks/pytest-9.0.3

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Sep 17, 2026

Copy link
Copy Markdown

Bumps pytest from 8.4.2 to 9.0.3.

Release notes

Sourced from pytest's releases.

9.0.3

pytest 9.0.3 (2026-04-07)

Bug fixes

  • #12444: Fixed pytest.approx which now correctly takes into account ~collections.abc.Mapping keys order to compare them.

  • #13634: Blocking a conftest.py file using the -p no: option is now explicitly disallowed.

    Previously this resulted in an internal assertion failure during plugin loading.

    Pytest now raises a clear UsageError explaining that conftest files are not plugins and cannot be disabled via -p.

  • #13734: Fixed crash when a test raises an exceptiongroup with __tracebackhide__ = True.

  • #14195: Fixed an issue where non-string messages passed to unittest.TestCase.subTest() were not printed.

  • #14343: Fixed use of insecure temporary directory (CVE-2025-71176).

Improved documentation

  • #13388: Clarified documentation for -p vs PYTEST_PLUGINS plugin loading and fixed an incorrect -p example.
  • #13731: Clarified that capture fixtures (e.g. capsys and capfd) take precedence over the -s / --capture=no command-line options in Accessing captured output from a test function <accessing-captured-output>.
  • #14088: Clarified that the default pytest_collection hook sets session.items before it calls pytest_collection_finish, not after.
  • #14255: TOML integer log levels must be quoted: Updating reference documentation.

Contributor-facing changes

  • #12689: The test reports are now published to Codecov from GitHub Actions. The test statistics is visible on the web interface.

    -- by aleguy02

9.0.2

pytest 9.0.2 (2025-12-06)

Bug fixes

  • #13896: The terminal progress feature added in pytest 9.0.0 has been disabled by default, except on Windows, due to compatibility issues with some terminal emulators.

    You may enable it again by passing -p terminalprogress. We may enable it by default again once compatibility improves in the future.

    Additionally, when the environment variable TERM is dumb, the escape codes are no longer emitted, even if the plugin is enabled.

  • #13904: Fixed the TOML type of the tmp_path_retention_count settings in the API reference from number to string.

  • #13946: The private config.inicfg attribute was changed in a breaking manner in pytest 9.0.0. Due to its usage in the ecosystem, it is now restored to working order using a compatibility shim. It will be deprecated in pytest 9.1 and removed in pytest 10.

... (truncated)

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    You can disable automated security fix PRs for this repo from the Security Alerts page.

@dependabot
chore(deps-dev): bump pytest from 8.4.2 to 9.0.3 in /scripts/benchmarks
09f11b3 
Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.4.2 to 9.0.3.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@8.4.2...9.0.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.0.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot Bot added dependencies Pull requests that update a dependency file python:uv Pull requests that update python:uv code labels Sep 17, 2026
@github-actions

github-actions Bot commented Sep 17, 2026 •
edited
Loading

Copy link
Copy Markdown

Prime Agent performance — completed

PR 09f11b3b compared with main c13e0311.

Overall: 31 regressed · 0 improved · 9 no clear change.

Metric Main This PR Change
Cold startup 1,238.5 ms 3,347.6 ms $\textcolor{#e5484d}{\textsf{↑ +2,109.1 ms (+170.29\%)}}$
Warm startup 806.6 ms 1,990.9 ms $\textcolor{#e5484d}{\textsf{↑ +1,184.3 ms (+146.82\%)}}$
Installation 13.65 s 20.29 s $\textcolor{#c75959}{\textsf{↑ +6.63 s (+48.59\%)}}$
Compressed release artifacts 72.49 MB 72.49 MB ≈ +0.0065 MB (+0.01%)
Installed footprint 577.65 MB 577.65 MB ≈ -0.0031 MB (-0.00%)
Idle memory, summed RSS 1,260.62 MB 1,249.38 MB ≈ -11.25 MB (-0.89%)

Python runtime

Metric Main This PR Change
Python kernel startup 117.6 ms 209.8 ms $\textcolor{#d84f52}{\textsf{↑ +92.2 ms (+78.35\%)}}$
Python cell round trip 0.436 ms 0.899 ms $\textcolor{#e5484d}{\textsf{↑ +0.463 ms (+106.23\%)}}$
Empty bash command 11.0 ms 21.8 ms $\textcolor{#e4494d}{\textsf{↑ +10.8 ms (+98.26\%)}}$
Bash git status 15.0 ms 31.9 ms $\textcolor{#e5484d}{\textsf{↑ +16.9 ms (+112.69\%)}}$
Bash 32 KiB output 10.0 ms 22.5 ms $\textcolor{#e5484d}{\textsf{↑ +12.4 ms (+124.23\%)}}$
35 cells / 9 shell calls 150.3 ms 304.3 ms $\textcolor{#e5484d}{\textsf{↑ +154.0 ms (+102.42\%)}}$
Python interrupt to done 1.220 ms 2.098 ms ≈ +0.878 ms (+71.98%)
Python state snapshot 24.6 ms 41.2 ms $\textcolor{#d25355}{\textsf{↑ +16.6 ms (+67.23\%)}}$
Python state restore 304.3 ms 560.1 ms $\textcolor{#dc4d51}{\textsf{↑ +255.8 ms (+84.05\%)}}$
Python idle RSS 36.51 MB 34.63 MB ≈ -1.88 MB (-5.15%)
Python RSS after pandas workload 98.68 MB 96.67 MB ≈ -2.01 MB (-2.04%)

UI interactions

Metric Main This PR Change
Resume large session (cold) 2,944.1 ms 6,651.7 ms $\textcolor{#e5484d}{\textsf{↑ +3,707.6 ms (+125.93\%)}}$
CPU, resume large session 4,930.0 ms 11,060.0 ms $\textcolor{#e5484d}{\textsf{↑ +6,130.0 ms (+124.34\%)}}$
Switch into large session 3,833.3 ms 8,751.1 ms $\textcolor{#e5484d}{\textsf{↑ +4,917.8 ms (+128.29\%)}}$
CPU, switch into large session 6,520.0 ms 16,210.0 ms $\textcolor{#e5484d}{\textsf{↑ +9,690.0 ms (+148.62\%)}}$
Open agents view from a session 154.0 ms 217.9 ms $\textcolor{#c25c5b}{\textsf{↑ +63.9 ms (+41.51\%)}}$
CPU, open agents view 290.0 ms 490.0 ms ≈ +200.0 ms (+68.97%)
Full agents roster, many sessions 4.04 s 5.30 s $\textcolor{#bc5f5e}{\textsf{↑ +1.26 s (+31.06\%)}}$
CPU, full agents roster 2.09 s 4.95 s $\textcolor{#e5484d}{\textsf{↑ +2.86 s (+136.84\%)}}$
Open another session from agents view 2,354.0 ms 3,787.2 ms $\textcolor{#ce5556}{\textsf{↑ +1,433.1 ms (+60.88\%)}}$
CPU, open from agents view 2,270.0 ms 4,950.0 ms $\textcolor{#e5484d}{\textsf{↑ +2,680.0 ms (+118.06\%)}}$
Reopen resident large session 420.6 ms 1,134.3 ms $\textcolor{#e5484d}{\textsf{↑ +713.7 ms (+169.66\%)}}$
CPU, reopen resident session 860.0 ms 2,100.0 ms ≈ +1,240.0 ms (+144.19%)
Open subagent session at depth 6 19,414.8 ms 25,434.0 ms $\textcolor{#bc5f5e}{\textsf{↑ +6,019.2 ms (+31.00\%)}}$
CPU, open subagent at depth 6 7,530.0 ms 16,510.0 ms $\textcolor{#e5484d}{\textsf{↑ +8,980.0 ms (+119.26\%)}}$
Open chain parent from agents view 3,604.5 ms 5,591.4 ms $\textcolor{#cb5758}{\textsf{↑ +1,986.9 ms (+55.12\%)}}$
CPU, open chain parent 2,900.0 ms 7,150.0 ms $\textcolor{#e5484d}{\textsf{↑ +4,250.0 ms (+146.55\%)}}$
Scheduled catalog, first request 1,302.9 ms 3,604.6 ms $\textcolor{#e5484d}{\textsf{↑ +2,301.7 ms (+176.65\%)}}$
CPU, scheduled catalog 2,330.0 ms 6,010.0 ms $\textcolor{#e5484d}{\textsf{↑ +3,680.0 ms (+157.94\%)}}$
Scheduled catalog, repeated request 824.0 ms 2,806.2 ms $\textcolor{#e5484d}{\textsf{↑ +1,982.1 ms (+240.54\%)}}$
CPU, repeated catalog 1,160.0 ms 4,010.0 ms $\textcolor{#e5484d}{\textsf{↑ +2,850.0 ms (+245.69\%)}}$
Cold worker with three catalog scans 1,168.9 ms 3,568.1 ms $\textcolor{#e5484d}{\textsf{↑ +2,399.2 ms (+205.25\%)}}$
CPU, cold worker and scans 3,450.0 ms 9,370.0 ms $\textcolor{#e5484d}{\textsf{↑ +5,920.0 ms (+171.59\%)}}$
UI memory after interactions 2,733.06 MB 2,692.85 MB ≈ -40.21 MB (-1.47%)

Sandbox cost: ~$0.1728 — no inference calls.
Run, logs, and downloadable raw results

Methodology and samples

Main resolved at 2026-09-17T21:00:02.594381+00:00. Harness c13e0311.
Linux x64, 4 vCPU, 8 GB RAM, 20 GB disk; region us.
Image: node:24-bookworm@sha256:be23f54a88d34e8824c741b19b91064094f92c1c97b194144bfc8b50d67258e2.
Stock tools, skills, daemon, and Python bootstrap enabled; fresh homes and a fixed Git fixture.
Onboarding is dismissed; the editor starts without a selected model or submitted prompt.
Medians shown. Arrows require a 20% timing/memory change plus absolute floors and IQR.
These practical noise floors are not a statistical significance test.
Cold means stopped Prime processes; OS filesystem caches are not flushed.
No model requests or credentials. Installation excludes build/setup time.
Installer tarballs use loopback; npm/Python downloads use the network with fresh caches.
Artifact size counts release tarballs; footprint after first use includes registry packages.
MB is decimal. Summed RSS can double-count shared pages; PSS is recorded when available.
Provisioning, setup, and build durations are recorded separately in the raw results.
Kernel probes use the installed JSONL runtime, outside the TUI/TypeScript host.
Per trial: 50 Python cells, 5 calls per shell case, and one 35-cell mix (9 git status calls).
Cell/shell values are batch means; other runtime timings are single operations.
State fixture: a 10,000-row × 8-column integer DataFrame and a 10,000-integer list.
Restore runs in a fresh kernel, including pandas imports; kernel startup is excluded.
Kernel RSS covers the isolated Python process; loaded RSS follows the pandas workload.
UI trials use a fresh fixture set: 194 top-level sessions including one ~40 MB transcript,
40 ledger fan-out children, and a 6-deep subagent chain (~46 spawn edges).
Large fixtures hold 1,999 complete triples (~5 MB JSONL); medium 119; subagents 399 each.
Interactions: cold --resume of a large session, warm /resume switch, left-arrow to agents view,
roster settle with many saved sessions, search-and-open of another large session,
reattaching to that resident session, opening the chain parent, and drilling to depth 6.
Readiness is the rendered transcript tail plus a confirmed editor echo.
CPU metrics sum utime+stime across the whole benchmark-user process tree per interaction.
UI memory sums RSS after the interactions; PTY byte counts are in the raw results.
A separate catalog fixture has 2,300 sessions, 2,298 edges, and 13 paused scheduled-job owners.
Catalog timings cover first/repeated reads and cold worker creation under three pending scans.
All expected jobs and owner metadata are checked; worker readiness excludes TUI rendering.
Costs estimate full sandbox lifetimes at configured rates, including setup and build.
Budget target: $1; not a billing cap. Performance changes are informational.
Failed or incomplete execution fails the workflow; saved artifacts remain available.
Each side stops a phase after 2 identical consecutive failures.
Skipped trials are not attempted samples. Warm startup requires a successful cold launch.

Metric Main successful/attempted PR successful/attempted Main spread PR spread
Cold startup 10/10 10/10 IQR 135.2 ms IQR 349.6 ms
Warm startup 10/10 10/10 IQR 101.3 ms IQR 286.8 ms
Installation 3/3 3/3 range 2.01 s range 0.91 s
Compressed release artifacts 1/1 1/1 — —
Installed footprint 1/1 1/1 — —
Idle memory, summed RSS 10/10 10/10 IQR 6.70 MB IQR 74.05 MB
Python kernel startup 10/10 10/10 IQR 17.3 ms IQR 42.6 ms
Python cell round trip 10/10 10/10 IQR 0.096 ms IQR 0.285 ms
Empty bash command 10/10 10/10 IQR 1.8 ms IQR 3.5 ms
Bash git status 10/10 10/10 IQR 1.5 ms IQR 7.1 ms
Bash 32 KiB output 10/10 10/10 IQR 1.2 ms IQR 4.7 ms
35 cells / 9 shell calls 10/10 10/10 IQR 23.7 ms IQR 56.1 ms
Python interrupt to done 10/10 10/10 IQR 0.250 ms IQR 1.580 ms
Python state snapshot 10/10 10/10 IQR 9.2 ms IQR 11.3 ms
Python state restore 10/10 10/10 IQR 23.1 ms IQR 97.7 ms
Python idle RSS 10/10 10/10 IQR 2.91 MB IQR 2.68 MB
Python RSS after pandas workload 10/10 10/10 IQR 3.01 MB IQR 2.73 MB
Resume large session (cold) 3/3 3/3 range 456.6 ms range 1,479.1 ms
CPU, resume large session 3/3 3/3 range 670.0 ms range 3,640.0 ms
Switch into large session 3/3 3/3 range 219.9 ms range 1,661.9 ms
CPU, switch into large session 3/3 3/3 range 350.0 ms range 880.0 ms
Open agents view from a session 3/3 3/3 range 11.7 ms range 29.7 ms
CPU, open agents view 3/3 3/3 range 250.0 ms range 210.0 ms
Full agents roster, many sessions 3/3 3/3 range 0.0062 s range 0.02 s
CPU, full agents roster 3/3 3/3 range 0.10 s range 0.25 s
Open another session from agents view 3/3 3/3 range 156.9 ms range 365.2 ms
CPU, open from agents view 3/3 3/3 range 460.0 ms range 680.0 ms
Reopen resident large session 3/3 3/3 range 40.3 ms range 446.1 ms
CPU, reopen resident session 3/3 3/3 range 200.0 ms range 1,300.0 ms
Open subagent session at depth 6 3/3 3/3 range 1,120.6 ms range 2,579.7 ms
CPU, open subagent at depth 6 3/3 3/3 range 1,490.0 ms range 3,940.0 ms
Open chain parent from agents view 3/3 3/3 range 123.1 ms range 1,199.8 ms
CPU, open chain parent 3/3 3/3 range 180.0 ms range 3,290.0 ms
Scheduled catalog, first request 3/3 3/3 range 253.2 ms range 538.1 ms
CPU, scheduled catalog 3/3 3/3 range 360.0 ms range 690.0 ms
Scheduled catalog, repeated request 3/3 3/3 range 58.6 ms range 1,345.6 ms
CPU, repeated catalog 3/3 3/3 range 250.0 ms range 1,640.0 ms
Cold worker with three catalog scans 3/3 3/3 range 116.8 ms range 1,838.0 ms
CPU, cold worker and scans 3/3 3/3 range 200.0 ms range 4,160.0 ms
UI memory after interactions 3/3 3/3 range 118.62 MB range 90.99 MB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

dependencies Pull requests that update a dependency file python:uv Pull requests that update python:uv code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

0 participants