feat: editor e2e #378

deltork · 2025-01-07T15:10:23Z

PR Goal?

Add tests for the studio editor

Fixes?

n/a

Feedback sought?

readiness check

Priority?

high

Tests added?

How to test?

Checkout this branch
Run studio-cli on your local machine
Run npx nx run-many --targets=serve,serve-fr,serve-es --projects=studio-web in the local branch
Wait until your browser opens, and you can confirm that studio is up and running
Run the tests npx nx e2e studio-web
Wait for the report

Test notes

Firefox tests are currently not 100% stable
Mobile Chrome test of alignment change is flaky in GitHub ci

Confidence?

high

Version change?

no, all changes are to support end-to-end testing

semanticdiff-com · 2025-01-07T15:10:26Z

Review changes with

Changed Files

File	Status
packages/studio-web/src/app/app.component.html	86% smaller
packages/studio-web/src/app/shared/download/download.service.ts	67% smaller
packages/studio-web/playwright.config.ts	35% smaller
packages/studio-web/tests/test-commands.ts	18% smaller
.github/workflows/end-to-end-tests.yml	Unsupported file format
README.md	Unsupported file format
packages/studio-web/tests/editor/check-page.spec.ts	0% smaller
packages/studio-web/tests/editor/edit-images.spec.ts	0% smaller
packages/studio-web/tests/editor/edit-translations.spec.ts	0% smaller
packages/studio-web/tests/editor/tour-editor.spec.ts	0% smaller
packages/studio-web/tests/editor/use-audio-toolbar.spec.ts	0% smaller
packages/studio-web/tests/fixtures/sentence-paragr.html	0% smaller
packages/studio-web/tests/studio-web/check-page-1.spec.ts	0% smaller
packages/studio-web/tests/test-cases.md	Unsupported file format

github-actions · 2025-01-07T15:13:22Z

PR Preview Action v1.5.0
Preview removed because the pull request was closed.
2025-01-09 20:20 UTC

joanise · 2025-01-07T16:53:51Z

We should document somewhere that we need to run npx playwright install --with-deps firefox chromium webkit before running npx nx e2e studio-web for the first time.

joanise · 2025-01-07T16:56:18Z

And also that npx nx e2e studio-web tests Studio-Web. The logical place is ### Testing/#### Studio-Web in the top-level README.md.

joanise · 2025-01-07T16:37:51Z

packages/studio-web/tests/test-commands.ts

+export const disablePlausible = async (page: Page) => {
+  //disable plausible
+  await page.evaluate(
+    async () => await window.localStorage.setItem("plausible_ignore", "true"),


nice and simple

joanise · 2025-01-07T16:38:23Z

packages/studio-web/tests/studio-web/check-page-1.spec.ts

@@ -1,9 +1,10 @@
 import { test, expect } from "@playwright/test";
-import { testText, defaultBeforeEach } from "../test-commands";
+import { testText, disablePlausible } from "../test-commands";


Nice, thanks for adding this!

joanise

Ready to merge, except for the top-level README.md documenting how to run these tests.

deltork · 2025-01-08T19:59:17Z

npx playwright install --with-deps firefox chromium webkit

It is part of the husky/post-install process. Should I make it post-checkout instead?

…d test

joanise · 2025-01-08T23:43:09Z

The updated readme looks nice, thanks!

joanise · 2025-01-09T17:01:32Z

npx playwright install --with-deps firefox chromium webkit

It is part of the husky/post-install process. Should I make it post-checkout instead?

Post install seems like the right time, as long as it's also documented (as you now did) because if I update an existing sandbox where I did the install before this was added, then I won't have done it.

joanise

The looks good to me.

The only thing I disagree with is going to 9 shards: we spend about 3 minutes in each shard setting up (install, spin up, etc), and in each shard we do at most 1 minute of work. It just feels like wasted resources. I would go back to 4 shards, maybe even 3, to keep the ratio of setup to work more reasonable. I know this will mean waiting a couple minutes longer for CI to finish, but I don't like all those CI minutes just spent installing.

Is it possible to have multiple threads in the same shard?
According to https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories the standard runner has 4 CPUs, so maybe it's possible to have just one job doing all the installation just once, and have 4 separate processes in the same job doing all the shards?

joanise · 2025-01-09T17:37:45Z

so maybe it's possible to have just one job doing all the installation just once, and have 4 separate processes in the same job doing all the shards?

Yes, I believe it is. I just tested and this works in CI:

      - name: background and wait
        run: |
          for shard in 1 2 3 4; do
            echo "shard $shard" && sleep 3 &
          done; wait
          echo "all done"

Replace the echo by the shard command, and then you could run the playwright merge reports command right after in the next step.

joanise · 2025-01-09T18:30:18Z

Oh, wait, playwriting accepts a --workers 4 option that does the same thing internally, much simpler. Never mind my manual splitting business!

deltork · 2025-01-09T20:18:02Z

Oh, wait, playwriting accepts a --workers 4 option that does the same thing internally, much simpler. Never mind my manual splitting business!

I have reduced the shards to 3 and increased the workers to 4.

deltork requested review from joanise and roedoejet January 7, 2025 15:10

joanise reviewed Jan 7, 2025

View reviewed changes

joanise approved these changes Jan 7, 2025

View reviewed changes

deltork added 9 commits January 8, 2025 16:42

test: disable plausible for tests

516057c

test: added test to check editor UI

fc9cafa

test: added test for editor editing images

2eabc12

test: refactored editor tests

5ccf43e

test: edded test for editor editing translations

c3a16e0

test: added tests for editor audio bar controls

72e371f

test: added test for the editor tour

ab4288a

test: added conditions for test to pass in ci

9261dca

docs: updated readme with instructions on how to run studio end-to-en…

9614658

…d test

deltork force-pushed the dev.del/feat-editor-e2e branch from 5cd5d9c to 9614658 Compare January 8, 2025 21:52

deltork force-pushed the dev.del/feat-editor-e2e branch from b7a86ab to 8fb71e5 Compare January 9, 2025 16:16

deltork requested a review from joanise January 9, 2025 16:44

joanise approved these changes Jan 9, 2025

View reviewed changes

ci: refactored end-to-end workflow

1813969

deltork force-pushed the dev.del/feat-editor-e2e branch from 8fb71e5 to 1813969 Compare January 9, 2025 19:39

deltork merged commit 44b9915 into main Jan 9, 2025
6 checks passed

deltork deleted the dev.del/feat-editor-e2e branch January 9, 2025 20:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: editor e2e #378

feat: editor e2e #378

deltork commented Jan 7, 2025

semanticdiff-com bot commented Jan 7, 2025 •

edited

Loading

github-actions bot commented Jan 7, 2025 •

edited

Loading

joanise commented Jan 7, 2025

joanise commented Jan 7, 2025

joanise Jan 7, 2025

joanise Jan 7, 2025

joanise left a comment

deltork commented Jan 8, 2025

joanise commented Jan 8, 2025

joanise commented Jan 9, 2025

joanise left a comment

joanise commented Jan 9, 2025

joanise commented Jan 9, 2025

deltork commented Jan 9, 2025

feat: editor e2e #378

feat: editor e2e #378

Conversation

deltork commented Jan 7, 2025

PR Goal?

Fixes?

Feedback sought?

Priority?

Tests added?

How to test?

Test notes

Confidence?

Version change?

semanticdiff-com bot commented Jan 7, 2025 • edited Loading

github-actions bot commented Jan 7, 2025 • edited Loading

joanise commented Jan 7, 2025

joanise commented Jan 7, 2025

joanise Jan 7, 2025

Choose a reason for hiding this comment

joanise Jan 7, 2025

Choose a reason for hiding this comment

joanise left a comment

Choose a reason for hiding this comment

deltork commented Jan 8, 2025

joanise commented Jan 8, 2025

joanise commented Jan 9, 2025

joanise left a comment

Choose a reason for hiding this comment

joanise commented Jan 9, 2025

joanise commented Jan 9, 2025

deltork commented Jan 9, 2025

semanticdiff-com bot commented Jan 7, 2025 •

edited

Loading

github-actions bot commented Jan 7, 2025 •

edited

Loading