Skip to content

chore(ci): Work on improving flaky Windows playwright smoke tests#1756

Merged
Tobbe merged 5 commits into
mainfrom
tobbe-chore-flaky-ci-win-smoke-tests
May 12, 2026
Merged

chore(ci): Work on improving flaky Windows playwright smoke tests#1756
Tobbe merged 5 commits into
mainfrom
tobbe-chore-flaky-ci-win-smoke-tests

Conversation

@Tobbe
Copy link
Copy Markdown
Member

@Tobbe Tobbe commented May 12, 2026

From run
25702624909
(PR #1754, Windows fragments smoke tests):

  • 10 tests total, 3 passed, 7 failed across all retries
  • The 3 passing tests use noJsBrowser to navigate to pre-rendered pages
  • Every page.goto() call fails with ERR_CONNECTION_REFUSED
  • page.waitForResponse() times out after 30 seconds
  • All 3 retries per test produce the same error — the server does not recover

@netlify
Copy link
Copy Markdown

netlify Bot commented May 12, 2026

Deploy Preview for cedarjs ready!

Name Link
🔨 Latest commit cae417f
🔍 Latest deploy log https://app.netlify.com/projects/cedarjs/deploys/6a0341227aa9a20008b634af
😎 Deploy Preview https://deploy-preview-1756--cedarjs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot added this to the chore milestone May 12, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 12, 2026

Greptile Summary

This PR applies the localhost127.0.0.1 fix uniformly across all 12 Playwright smoke-test configs and the 3 test files that hard-coded localhost in URL assertions, eliminating the Windows IPv6 resolution issue (localhost::1) that caused ERR_CONNECTION_REFUSED failures in CI. An investigation doc is added tracking the full history of the flakiness and the remaining open questions (server crash PID change, Apollo error during /double prerender).

  • 12 Playwright configs (baseURL + webServer.url) changed from http://localhost:… to http://127.0.0.1:…, covering every smoke-test suite.
  • 3 test/helper files updated to match; a cookie domain that incorrectly included the port (localhost:8910) is corrected to 127.0.0.1.

Confidence Score: 5/5

Safe to merge — the change is a mechanical host-string replacement with no logic alterations.

All changes are straightforward string substitutions in test config and test assertion files. The cookie domain fix is also correct — the old value included a port, which is invalid per the cookie spec. No runtime logic, no production paths, and no schema changes are touched.

serve/playwright.config.ts, prerender/playwright.config.ts, and fragments-serve/playwright.config.ts each contain an orphaned comment whose corresponding timeout property was reverted.

Important Files Changed

Filename Overview
docs/implementation-plans/flaky-windows-smoke-tests-investigation.md New investigation doc tracking the Windows ERR_CONNECTION_REFUSED flakiness; updated with scope expansion, timeout revert, and notes on Vite/CLI crash modes. Factually accurate.
tasks/smoke-tests/shared/common.ts URL assertions updated from localhost:8910 to 127.0.0.1:8910 to match the new baseURL in all configs.
tasks/smoke-tests/rsc-kitchen-sink/tests/rsc-kitchen-sink.spec.ts Cookie domain corrected from localhost:8910 (invalid — port in domain) to 127.0.0.1; import extension made explicit; await expect(await …) anti-pattern cleaned up.
tasks/smoke-tests/storybook/tests/storybook.spec.ts All six toHaveURL assertions updated from localhost:7910 to 127.0.0.1:7910 to match the updated baseURL.
tasks/smoke-tests/dev/tests/authChecks.spec.ts Login redirect URL assertion updated from localhost:8910 to 127.0.0.1:8910.
tasks/smoke-tests/serve/playwright.config.ts localhost127.0.0.1 in baseURL and webServer.url; orphaned // Give the server more time to start on CI comment left without a matching timeout property.
tasks/smoke-tests/fragments-serve/playwright.config.ts localhost127.0.0.1; same orphaned comment as serve/playwright.config.ts.
tasks/smoke-tests/prerender/playwright.config.ts localhost127.0.0.1; same orphaned comment as serve/playwright.config.ts.
tasks/smoke-tests/storybook/playwright.config.ts localhost:7910127.0.0.1:7910 in both baseURL and webServer.url.
tasks/smoke-tests/live/playwright.config.mts localhost127.0.0.1 for both baseURL (8910) and webServer.url (8911).

Reviews (5): Last reviewed commit: "dbAuth secret gen" | Re-trigger Greptile

@nx-cloud
Copy link
Copy Markdown

nx-cloud Bot commented May 12, 2026

🤖 Nx Cloud AI Fix

Ensure the fix-ci command is configured to always run in your CI pipeline to get automatic fixes in future runs. For more information, please see https://nx.dev/ci/features/self-healing-ci


View your CI Pipeline Execution ↗ for commit 42bd468

Command Status Duration Result
nx run-many -t build:pack --exclude create-ceda... ✅ Succeeded 5s View ↗
nx run-many -t build ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2026-05-12 13:23:52 UTC

@Tobbe Tobbe merged commit 9d21584 into main May 12, 2026
33 checks passed
@Tobbe Tobbe deleted the tobbe-chore-flaky-ci-win-smoke-tests branch May 12, 2026 16:34
@github-actions
Copy link
Copy Markdown

The changes in this PR are now available on npm.

Try them out by running yarn cedar upgrade -t 5.0.0-canary.2324

Or try it in a new app with yarn dlx create-cedar-app@5.0.0-canary.2324

Tobbe added a commit that referenced this pull request May 12, 2026
)

From run

[25702624909](https://github.com/cedarjs/cedar/actions/runs/25702624909/job/75468866374)
(PR #1754, Windows fragments smoke tests):

- 10 tests total, 3 passed, 7 failed across all retries
- The 3 passing tests use `noJsBrowser` to navigate to pre-rendered
pages
- Every `page.goto()` call fails with `ERR_CONNECTION_REFUSED`
- `page.waitForResponse()` times out after 30 seconds
- All 3 retries per test produce the same error — the server does not
recover
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant