Fix running playwright test with npm and electron.

[tsmaeder]

What it does

This PR launches the electron playwright tests with the installed electron version. It also introduces an environment variable THEIA_NO_SPLASH to prevent showing a splash screen (which breaks the tests).

Fixes #14763

Contributed on behalf of STMicroelectronics

How to test

Run the playwright tests with electron. Note that we don't expect all of the tests to pass for electron.

Follow-ups

Breaking changes

• This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Contributed on behalf of STMicroelectronics

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[planger]

Thank you very much for looking into this issue!

I tried to run the Electron tests locally, but for some reason it failed for me:

~/Git/OpenSource/Theia/theia/examples/playwright (pr/tsmaeder/14764 ✗) echo $THEIA_NO_SPLASH      
true
~/Git/OpenSource/Theia/theia/examples/playwright (pr/tsmaeder/14764 ✗) npm run ui-tests-electron

> @theia/[email protected] ui-tests-electron
> npm run build && cross-env USE_ELECTRON=true playwright test --config=./configs/playwright.config.ts


> @theia/[email protected] build
> theiaext build && npm run playwright:install

$ ts-clean-dangling && tsc --build

> @theia/[email protected] playwright:install
> playwright install chromium


Running 94 tests using 1 worker

  ✘  1 …ts/theia-app.test.ts:28:9 › Theia Application › should load and should show main content panel (686ms)
Launching Electron with options: {"args":["../electron","--no-cluster","--app-project-path=../electron","--plugins=local-dir:../../plugins","/tmp/cloud-ws-RLWK0n"],"env":{"THEIA_NO_SPLASH":"true"}}
  -  2 …ication-shell.test.ts:53:10 › Theia Application Shell › should open and close a text editor many times
  ✘  3 …plorer-view.test.ts:48:9 › Theia Explorer View › should be visible and active after being opened (0ms)
Launching Electron with options: {"args":["../electron","--no-cluster","--app-project-path=../electron","--plugins=local-dir:../../plugins","/tmp/cloud-ws-BXG7f4"],"env":{"THEIA_NO_SPLASH":"true"}}
  -  4 …r-view.test.ts:54:9 › Theia Explorer View › should be opened at the left and have the title 'Explorer'
  -  5 …sts/theia-explorer-view.test.ts:60:9 › Theia Explorer View › should be possible to close and reopen it
  -  6 … show one folder named "sampleFolder", one named "sampleFolderCompact" and one file named "sample.txt"
  -  7 …est.ts:98:9 › Theia Explorer View › should provide file stat node by single path fragment "sample.txt"
  -  8 …le stat nodes that can define whether they are collapsed or not and that can be expanded and collapsed
  -  9 …should provide file stat node by path "sampleFolder/sampleFolder1/sampleFolder1-1/sampleFile1-1-1.txt"
  -  10 … › should be able to check if compact folder "sampleFolderCompact/nestedFolder1/nestedFolder2" exists
  -  11 …at node by path of compact folder "sampleFolderCompact/nestedFolder1/nestedFolder2/sampleFile1-1.txt"
  -  12 …ts/theia-explorer-view.test.ts:139:9 › Theia Explorer View › should open context menu on "sample.txt"
  -  13 ../../src/tests/theia-explorer-view.test.ts:155:9 › Theia Explorer View › should rename "sample.txt"
  -  14 …st.ts:162:9 › Theia Explorer View › should open context menu on nested folder segment "nestedFolder1"
  -  15 …› Theia Explorer View › should rename compact folder "sampleFolderCompact" to "sampleDirectoryCompact
  -  16 …heia Explorer View › should delete nested folder "sampleDirectoryCompact/nestedFolder1/nestedFolder2"
  -  17 …ts:196:10 › Theia Explorer View › should delete compact folder "sampleDirectoryCompact/nestedFolder1"
  -  18 …ests/theia-explorer-view.test.ts:205:9 › Theia Explorer View › open "sample.txt" via the context menu
  ✘  19 …getting-started.test.ts:37:9 › Theia Welcome Page › New File... entry should create a new file. (0ms)
Launching Electron with options: {"args":["../electron","--no-cluster","--app-project-path=../electron","--plugins=local-dir:../../plugins","/tmp/cloud-ws-Uu11TK"],"env":{"THEIA_NO_SPLASH":"true"}}
     20 ../../src/tests/theia-main-menu.test.ts:39:9 › Theia Main Menu › should show the main menu bar
Launching Electron with options: {"args":["../electron","--no-cluster","--app-project-path=../electron","--plugins=local-dir:../../plugins","/tmp/cloud-ws-9KrnEW"],"env":{"THEIA_NO_SPLASH":"true"}}
^C

  1) ../../src/tests/theia-app.test.ts:28:9 › Theia Application › should load and should show main content panel 

    Error: electron.launch: Process failed to launch!

       at ../../src/theia-app-loader.ts:110

      108 |         const playwrightOptions = toPlaywrightOptions(launchOptions, workspace);
      109 |         console.log(`Launching Electron with options: ${JSON.stringify(playwrightOptions)}`);
    > 110 |         const electronApp = await electron.launch(playwrightOptions);
          |                             ^
      111 |         const page = await electronApp.firstWindow();
      112 |
      113 |         const appFactory = theiaAppFactory<T>(factory);

        at Object.load (/home/philip/Git/OpenSource/Theia/theia/examples/playwright/src/theia-app-loader.ts:110:29)
        at /home/philip/Git/OpenSource/Theia/theia/examples/playwright/src/tests/theia-app.test.ts:29:15

    TypeError: Cannot read properties of undefined (reading 'page')

      23 |
      24 |     test.afterAll(async () => {
    > 25 |         await app.page.close();
         |                   ^
      26 |     });
      27 |
      28 |     test('should load and should show main content panel', async ({ playwright, browser }) => {

        at /home/philip/Git/OpenSource/Theia/theia/examples/playwright/src/tests/theia-app.test.ts:25:19

  2) ../../src/tests/theia-explorer-view.test.ts:48:9 › Theia Explorer View › should be visible and active after being opened 

    Error: electron.launch: Process failed to launch!

       at ../../src/theia-app-loader.ts:110

      108 |         const playwrightOptions = toPlaywrightOptions(launchOptions, workspace);
      109 |         console.log(`Launching Electron with options: ${JSON.stringify(playwrightOptions)}`);
    > 110 |         const electronApp = await electron.launch(playwrightOptions);
          |                             ^
      111 |         const page = await electronApp.firstWindow();
      112 |
      113 |         const appFactory = theiaAppFactory<T>(factory);

        at Object.load (/home/philip/Git/OpenSource/Theia/theia/examples/playwright/src/theia-app-loader.ts:110:29)
        at /home/philip/Git/OpenSource/Theia/theia/examples/playwright/src/tests/theia-explorer-view.test.ts:31:15

    TypeError: Cannot read properties of undefined (reading 'page')

      43 |
      44 |     test.afterAll(async () => {
    > 45 |         await app.page.close();
         |                   ^
      46 |     });
      47 |
      48 |     test('should be visible and active after being opened', async () => {

        at /home/philip/Git/OpenSource/Theia/theia/examples/playwright/src/tests/theia-explorer-view.test.ts:45:19

Is there something I'm doing wrong?

I've built the root with npm run build:electron before and npm run start:electron works fine.

[tsmaeder]

@planger does the splash screen show up? Do you have the plugins downloaded?

[planger]

Yes, I've downloaded the plugins.

I confirmed that the splash screen doesn't show up in I run npm run start:electron and the app starts. But when executing the UI tests with electron, no application seems to come up whatsoever.

[tsmaeder]

When copying the parameters from playwright electron launcher, the launch fails because of the well known sandbox problem. Adding --no-sandbox fixes the problem when launching electron on the command line. Still doesn't help with the tests, though. But I suspect launching on a recent linux distro has never worked at all without -no-sandbox.

[tsmaeder]

It seems the problem I still have is with nvm: the ouput from launching the electron script says /usr/bin/env 'node': no such file or directory. The same happens when I do sudo /usr/bin/env node. nvm only puts nodejs on the path of the current terminal session. It seems the way playwright launches the session does not include the current node installation in the path and therefore node_modules/.bin/electron fails to launch.

[planger]

I can confirm that Electron UI tests now work on Ubuntu 22.04. Thank you! 🥳

[planger]

Thank you, works fine on Linux!