Failing to launch artillery with typescript playwright configuration

[arthurzenika]

Thank you for filing a bug report! 🐛 Please provide a short summary of the bug,
along with any information you feel relevant to replicating it.
\-->

Version info:

Artillery: 2.0.4
Node.js:   v16.13.1
OS:        linux

Running this command:

❯ artillery run artillery.yml
▲ [WARNING] "../package.json" should be marked as external for use with "require.resolve" [require-resolve-not-external]
  
                node_modules/@playwright/test/node_modules/playwright/lib/index.js:69:64:
                        69 │ ...nalStackPrefix)(path.dirname(require.resolve('../package.json')));
                                    ╵                                                 ~~~~~~~~~~~~~~~~~
  
▲ [WARNING] "playwright-core/package.json" should be marked as external for use with "require.resolve" [require-resolve-not-external]
  
                node_modules/@playwright/test/node_modules/playwright/lib/util.js:56:67:
                        56 │ ....default.dirname(require.resolve('playwright-core/package.json'));
                                    ╵                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
▲ [WARNING] "playwright/lib/transform/esmLoader" should be marked as external for use with "require.resolve" [require-resolve-not-external]
  
                node_modules/@playwright/test/node_modules/playwright/lib/util.js:278:121:
                        278 │ ...uire.resolve('playwright/lib/transform/esmLoader')).toString()...
                                        ╵                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
▲ [WARNING] "./chromium/appIcon.png" should be marked as external for use with "require.resolve" [require-resolve-not-external]
  
                node_modules/@playwright/test/node_modules/playwright-core/lib/server/launchApp.js:63:67:
                        63 │ ...ault.promises.readFile(require.resolve('./chromium/appIcon.png'));
                                    ╵                                           ~~~~~~~~~~~~~~~~~~~~~~~~
  
▲ [WARNING] "./loader" should be marked as external for use with "require.resolve" [require-resolve-not-external]
  
                node_modules/@playwright/test/node_modules/playwright-core/lib/server/electron/electron.js:152:56:
                        152 │ ...    electronArguments.unshift('-r', require.resolve('./loader'));
                                        ╵                                                        ~~~~~~~~~~
  
Bundled Typescript file into JS. New processor path: /tmp/playwright.config-1706262578798.js
Test run id: twknc_b4jp45me9zcft3wczy8dgqy47fxna_7qch
⠏ worker error, id: 2 Error: Cannot find module 'playwright-core/package.json'
Require stack:
- /tmp/playwright.config-1706262578798.js
- /home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/runner.js
- /home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/node_modules/@artilleryio/int-core/index.js
- /home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/lib/artillery-global.js
- /home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/lib/platform/local/worker.js
                at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
                at Function.resolve (node:internal/modules/cjs/helpers:108:19)
                at null.node_modules/@playwright/test/node_modules/playwright/lib/util.js (/home/arthur/src/project/node_modules/@playwright/test/node_modules/playwright/lib/util.js:56:52)
                at __require (/tmp/playwright.config-1706262578798.js:8:50)
                at null.node_modules/@playwright/test/node_modules/playwright/lib/matchers/matchers.js (/home/arthur/src/project/node_modules/@playwright/test/node_modules/playwright/lib/matchers/matchers.js:31:13)
                at __require (/tmp/playwright.config-1706262578798.js:8:50)
                at null.node_modules/@playwright/test/node_modules/playwright/lib/matchers/expect.js (/home/arthur/src/project/node_modules/@playwright/test/node_modules/playwright/lib/matchers/expect.js:10:17)
                at __require (/tmp/playwright.config-1706262578798.js:8:50)
                at null.node_modules/@playwright/test/node_modules/playwright/lib/common/testType.js (/home/arthur/src/project/node_modules/@playwright/test/node_modules/playwright/lib/common/testType.js:9:15)
                at __require (/tmp/playwright.config-1706262578798.js:8:50) {
        code: 'MODULE_NOT_FOUND',
        requireStack: [
                '/tmp/playwright.config-1706262578798.js',
                '/home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/runner.js',
                '/home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/node_modules/@artilleryio/int-core/index.js',
                '/home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/lib/artillery-global.js',
                '/home/arthur/.asdf/installs/nodejs/16.13.1/lib/node_modules/artillery/lib/platform/local/worker.js'
        ]
}

I expected to see this happen:

artillery works with the flows.js demo, but fails when I try to use my existing playwright tests. For "normal" playwright I use npx playwright test. The reason I'm using v16.13.1 is that the playwright version used expects this version of node, not a more recent one (I have a hunch this might the problem, so will look into seeing if I can upgrade that version).

Instead, this happened:

The above traceback tells me I'm missing some part of the setup.

Files being used:
❯ cat artillery.yml
File: artillery.yml

config:
    target: https://example.org/
    # https://www.artillery.io
    # Load the Playwright engine:
    engines:
        playwright: {}
    # Path to JavaScript file that defines Playwright test functions
    #processor: "./flows.js" #demo
    processor: "./playwright.config.ts"
        #processor: "./src/scenarios/portail/testsAcceptation.spec.ts"
scenarios:
    - engine: playwright
        testFunction: "helloFlow"

playwright.config.ts :

File: playwright.config.ts
import {devices, PlaywrightTestConfig} from '@playwright/test';
  
const config: PlaywrightTestConfig = {
        testDir: './src/scenarios',
        outputDir: './test-results',
[snip]
};
export default config;

[bernardobridge]

Hey @arthurzenika 👋

This is a known limitation of Typescript support right now, and it's due to using @playwright/test in your processor. It's listed here:
https://www.artillery.io/docs/reference/engines/playwright#known-limitations

We'll work on providing workarounds for this in the near future, but for now you'll have to either not use that package or use Javascript.

On a side note, Artillery doesn't use the Playwright Test runner, so you wouldn't be setting configuration with a playwright.config.ts file. Configuration is set under config.engines.playwright, as explained here.

[arthurzenika]

Wow, thanks for the instant response, sorry I didn't understand the link between my traceback and the "known limitations" section.

Is there an issue where I can follow the progress on the workarounds ?

[bernardobridge]

Wow, thanks for the instant response, sorry I didn't understand the link between my traceback and the "known limitations" section.

That's good feedback @arthurzenika - we'll make it more clear in the docs the kind of errors that can appear.

Is there an issue where I can follow the progress on the workarounds ?

Not currently, but when we start implementing them, we'll make sure to post in relevant threads. In the meantime I've moved this to a discussion so others can see it's a known limitation, and I'll post here when we have more information about possible workarounds or solutions.

Thanks!

[bernardobridge]

Hey folks / @arthurzenika 👋

The new Artillery release (v2.0.6) contains improvements to the Typescript support.

Some documentation:

• https://www.artillery.io/docs/reference/test-script#preventing-bundling-of-typescript-packages
• https://www.artillery.io/docs/reference/engines/playwright#known-limitations

[AlejandroVisentini]

Hi @bernardobridge. I need to set http headers for CloudFare client-id and client-secret variables. It's working setting them up in the playwright.config.ts file through

use: {
    extraHTTPHeaders: {
      'CF-Access-Client-Id': 'xxxxx',
      'CF-Access-Client-Secret': 'xxxxx',
    },
  },

but it doesn't work when configuring those variables in the artillery yml file:

  engines:
    playwright:
      trace: true
      launchOptions:
        headless: false
      contextOptions:
        extraHTTPHeaders: {
          CF-Access-Client-Id: "xxxxx",
          CF-Access-Client-Secret: "xxxxx"
        }

also tried

extraHTTPHeaders: 
          CF-Access-Client-Id: "xxxxx",
          CF-Access-Client-Secret: "xxxxx"

extraHTTPHeaders: 
          CF-Access-Client-Id: 'xxxxx'
          CF-Access-Client-Secret: 'xxxxx'

extraHTTPHeaders: 
          'CF-Access-Client-Id': 'xxxxx'
          'CF-Access-Client-Secret': 'xxxxx'

extraHTTPHeaders: 
          CF-Access-Client-Id: xxxxx
          CF-Access-Client-Secret: xxxxx

I'm working on Windows. Could you help me please?