[py] rerun tests with debugging

[titusfortner]

User description

🔗 Related Issues

Implements #16879 for Python

💥 What does this PR do?

Turns on debugging for reruns

You can see the difference in logging here: https://github.com/SeleniumHQ/selenium/actions/runs/20899801802/job/60044210359?pr=16892

🔄 Types of changes

• New feature (non-breaking change which adds functionality and tests!)

---

PR Type

Enhancement

---

Description

• Enable debugging output for Python test reruns

• Add rerun-with-debug flag to remote and browser test workflows

• Improves debugging visibility for flaky test failures

---

Diagram Walkthrough

flowchart LR
  CI["CI Workflows"]
  RemoteTests["Remote Tests Job"]
  BrowserTests["Browser Tests Job"]
  DebugFlag["rerun-with-debug: true"]
  CI --> RemoteTests
  CI --> BrowserTests
  DebugFlag --> RemoteTests
  DebugFlag --> BrowserTests

Loading

File Walkthrough

	Relevant files
Enhancement	ci-python.yml Add debug flag to Python test rerun workflows                        .github/workflows/ci-python.yml Added rerun-with-debug: true parameter to remote-tests job Added rerun-with-debug: true parameter to browser-tests job Enables debugging output when tests are rerun due to flakiness +2/-0

---

[qodo-code-review]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	Debug log exposure Description: Enabling rerun-with-debug: true in CI may increase log verbosity and could inadvertently expose sensitive data (e.g., environment variables, tokens, or system details) in GitHub Actions logs during reruns, requiring verification of log redaction and secret-masking behavior. ci-python.yml [81-102] Referred Code rerun-with-debug: true run: | bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-remote browser-tests: name: Browser Tests needs: build uses: ./.github/workflows/bazel.yml strategy: fail-fast: false matrix: browser: [chrome, firefox, chrome-bidi, edge] os: [windows] include: - browser: safari os: macos with: name: Integration Tests (${{ matrix.browser }}) browser: ${{ matrix.browser }} os: ${{ matrix.os }} rerun-with-debug: true ... (clipped 1 lines)
Ticket Compliance
🟡	🎫 #1234 🔴 Ensure that click() in Selenium 2.48 triggers JavaScript contained in a link's href (regression vs 2.47.1), specifically for Firefox (reported on Firefox 42.0). Provide/ensure a reproducible test case where an alert is triggered on click, and make it pass on affected versions.
🟡	🎫 #5678 🔴 Fix or mitigate repeated ChromeDriver instantiation failures that result in "Error: ConnectFailure (Connection refused)" on Ubuntu/Chrome (Selenium 3.9.0, Chrome 65, ChromeDriver 2.35). Ensure that subsequent ChromeDriver instances after the first can be created without connection-refused errors.
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
⚪	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Debug output exposure: Enabling rerun-with-debug: true may increase verbosity and potentially expose internal stack traces or sensitive details in CI logs depending on how the downstream workflow implements debug reruns. Referred Code rerun-with-debug: true run: | bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-remote browser-tests: name: Browser Tests needs: build uses: ./.github/workflows/bazel.yml strategy: fail-fast: false matrix: browser: [chrome, firefox, chrome-bidi, edge] os: [windows] include: - browser: safari os: macos with: name: Integration Tests (${{ matrix.browser }}) browser: ${{ matrix.browser }} os: ${{ matrix.os }} rerun-with-debug: true Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Potential sensitive logs: Turning on debug for reruns could cause secrets, tokens, or other sensitive data to be printed to CI logs if any tests or tooling log such values at debug level. Referred Code rerun-with-debug: true run: | bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-remote browser-tests: name: Browser Tests needs: build uses: ./.github/workflows/bazel.yml strategy: fail-fast: false matrix: browser: [chrome, firefox, chrome-bidi, edge] os: [windows] include: - browser: safari os: macos with: name: Integration Tests (${{ matrix.browser }}) browser: ${{ matrix.browser }} os: ${{ matrix.os }} rerun-with-debug: true Learn more about managing compliance generic rules or creating your own custom rules
Update

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-code-review]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Use Bazel flag for debug output Add the --test_output=errors flag to the bazel test command to capture debug logs from flaky tests directly, which is more efficient than using a separate workflow-level rerun mechanism. .github/workflows/ci-python.yml [82-83] run: | - bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-remote + bazel test --local_test_jobs 1 --flaky_test_attempts 3 --test_output=errors //py:test-remote Apply / Chat Suggestion importance[1-10]: 8 __ Why: The suggestion provides a more standard and efficient solution by using Bazel's native --test_output=errors flag, which avoids potentially redundant retry logic at the workflow level and simplifies the CI configuration.	Medium
Learned best practice	Centralize repeated workflow inputs Avoid repeating rerun-with-debug: true across jobs by defining a shared YAML anchor (or similar shared config) and merging it into each with: block. .github/workflows/ci-python.yml [78-103] +x-bazel-with: &bazel_with + rerun-with-debug: true + +... with: + <<: *bazel_with name: Integration Tests Remote browser: yes - rerun-with-debug: true run: | bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-remote ... with: + <<: *bazel_with name: Integration Tests (${{ matrix.browser }}) browser: ${{ matrix.browser }} os: ${{ matrix.os }} - rerun-with-debug: true run: | bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-${{ matrix.browser }} [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 6 __ Why: Relevant best practice - Reduce duplication by centralizing shared behavior/configuration (avoid repeating identical workflow inputs in multiple jobs).	Low
Update

[navin772]

LGTM!