Skip to content

TKT-022: Run full test suite on PRs, not just smoke — prevent broken main#1056

Open
chrismcdermut wants to merge 2 commits intomainfrom
PRLT-1225/feat/enforce-tests-in-eve
Open

TKT-022: Run full test suite on PRs, not just smoke — prevent broken main#1056
chrismcdermut wants to merge 2 commits intomainfrom
PRLT-1225/feat/enforce-tests-in-eve

Conversation

@chrismcdermut
Copy link
Copy Markdown
Owner

Summary

Resolves TKT-022: Run full test suite on PRs, not just smoke — prevent broken main

Description

Problem

PR CI only runs smoke tier (e2e-tests (22, smoke)). Post-merge CI runs full suite (7 shards: pmo, execution, agent-flows, json-mode, integrations, standalone, infrastructure).

This means test failures can land on main undetected. We just had 17 tests break on main from merged PRs that passed PR CI.

Solution

Run full test suite on PRs, not just smoke. Options to keep it fast:

Option A: Full suite on all PRs

  • Run all 7 e2e shards + unit tests on every PR
  • Pro: catches everything
  • Con: slower CI (~3-4 min more per PR), more GitHub Actions minutes

Option B: Full suite on merge (merge queue)

  • Enable GitHub merge queue
  • Queue tests the merged result with full suite before landing
  • PR CI stays fast (smoke only)
  • Pro: catches merge-time issues, batches PRs
  • Con: adds latency between "PR approved" and "merged"

Option C: Full suite on PRs touching test-adjacent files

  • Detect which files changed, run relevant shards
  • Pro: targeted, fast for most PRs
  • Con: complex to maintain shard-to-file mapping

Recommendation

Option A is simplest. CI is already ~5 min for smoke. Full suite adds ~3 min. Total ~8 min is acceptable.

Files

  • .github/workflows/tests.yml — change PR tier from smoke to full

External Issue Context

Changes

  • 7df57e7 feat(PRLT-1225): add test enforcement regression tests — PreToolUse hook, prompt guidance, script validation
  • 9f60ce6 feat(PRLT-1225): add test enforcement: CLAUDE.md rules, agent prompt guidance, PreToolUse hook

Test Plan

  • Tests pass locally
  • Manual testing completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant