GTest test listing incorrectly passes --help

[Trass3r]

Checklist

• [/] The issue is about this extension and NOT about a fork.
• [/] Check the known issues list.
• [/] The latest version of the extension was used.
• [/] It is not related to remote-vscode

Describe the bug

For whatever reason it suddenly adds --help to the spawnargs when trying to get the test list, which prevents actual test list output. This used to work. I checked manually that test listing incl. xml works when run without --help.

"testMate.cpp.test.advancedExecutables": [
		{
			"pattern": "/build/**/*Test",
			"cwd": "/build/",
			"gtest": {"ignoreTestEnumerationStdErr": true},
			"executionWrapper": {"path": "/bin/bash", "args": ["/build/runtest.sh", "${cmd}", "${argsFlat}"]},
		}
	]

[INFO] reloadTests GoogleTest Promise { undefined } ..
[INFO] coudnt use cache [Error: ENOENT: no such file or directory, stat...
[INFO] discovering tests /build/Test [
  '--gtest_list_tests',
  '--gtest_output=xml:/build/Test.TestMate.testListCache.xml'
] /build
[INFO] Couldn't parse output file. Possibly it is an older version of Google Test framework. It is trying to parse the output
[INFO] GoogleTest._reloadFromStdOut error [Error: Wrong test list.
	at u._reloadFromString (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:360536)
	at u._reloadChildren (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:362369)
	at async /home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:251255
	at async /home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:272873
	at async e.ExecutableConfig.load (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:273640)
	at async Promise.all (index 0)
	at async o.load (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:284723)
	at async e.SharedVariables._shared.u.SharedVariables.e [as loadWithTask] (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:308086)
	at async e.TestAdapter.load (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:313121)] {
....
  stdout:...
    'This program contains tests written using Google Test. You can use the\n' +
    'following command line flags to control its behavior:\n' +
    '\n' +...
    'Except for --gtest_list_tests, you can alternatively set the corresponding\n' +
    'environment variable of a flag (all letters in upper-case). For example, to\n' +
    'disable colored text output, you can either specify --gtest_color=no or set\n' +
    'the GTEST_COLOR environment variable to no.\n' +
    '\n' +
    'For more information, please read the Google Test documentation at\n' +
    'https://github.com/google/googletest/. If you find a bug in Google Test\n' +
    '(not one in your own code or tests), please report it to\n' +
    '<[email protected]>.\n' +...
  stderr: ....,
  status: 0,
  signal: null,
  error: undefined,
  closed: true,
  process: ChildProcess {
    _events: [Object: null prototype] { error: [Function], close: [Function] },
    _eventsCount: 2,
    _maxListeners: undefined,
    _closesNeeded: 3,
    _closesGot: 3,
    connected: false,
    signalCode: null,
    exitCode: 0,
    killed: false,
    spawnfile: '/bin/bash',
    _handle: null,
    spawnargs: [
      '/bin/bash',
      '/build/runtest.sh',
      '/build/Test',
      '--help',
      '--gtest_list_tests',
      '--gtest_output=xml:/build/Test.TestMate.testListCache.xml'
    ],
[...]
[WARN] Couldn't load executable [Error: Wrong test list.
	at u._reloadFromString (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:360536)
	at u._reloadChildren (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:362369)
	at async /home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:251255
	at async /home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:272873
	at async e.ExecutableConfig.load (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:273640)
	at async Promise.all (index 0)
	at async o.load (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:284723)
	at async e.SharedVariables._shared.u.SharedVariables.e [as loadWithTask] (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:308086)
	at async e.TestAdapter.load (/home/vscode/.vscode-server/extensions/matepek.vscode-catch2-test-adapter-3.6.5/out/dist/main.js:2:313121)] u {
...
    _executables: [
      {
        _shared: [Object],
        _pattern: '/build/**/*Test',
        _name: undefined,
        _description: undefined,
        _cwd: '/build/',
        _env: undefined,
        _envFile: undefined,
        _dependsOn: [],
        _runTask: [Object],
        _parallelizationLimit: 8,
        _strictPattern: undefined,
        _markAsSkipped: undefined,
        _executionWrapper: [Object],
        _sourceFileMap: {},
        _catch2: {},
        _gtest: [Object],
        _doctest: {},
        _gbenchmark: {},
        _disposed: false,
        _disposables: [Array],
        _runnables: Map {},
        _lastEventArrivedAt: Map {}
      }
    ],
[...]
    _frameworkSpecific: { testGrouping: undefined, ignoreTestEnumerationStdErr: true },
    markAsSkipped: false,
    runTask: { before: [], beforeEach: [], after: [], afterEach: [] },
    spawner: s {
      _executor: '/bin/bash',
      _args: [
        '/build/runtest.sh',
        '${cmd}',
        '${argsFlat}',
        '--help',
        '--gtest_list_tests',
        '--gtest_output=xml:/build/Test.TestMate.testListCache.xml'
      ],

[matepek]

This does not add up. Did you delete lines at ....?

[Trass3r]

Yeah I highlighted the important parts.
It definitely picks up args from prior runs. After I created the testlistcache manually I saw the test in test explorer and when running it:

  spawner: s {
    _executor: '/bin/bash',
    _args: [
      '/build/runtest.sh',
      '${cmd}',
      '${argsFlat}',
      '--help',
      '--gtest_list_tests',
      '--gtest_output=xml:/build/Test.TestMate.testListCache.xml',
      '--gtest_color=no',
      '--gtest_filter=TestName',
      '--gtest_also_run_disabled_tests'
    ],

So --help is from the test framework check, next 2 are from test discovery and then it appends the runner args.
It may have been introduced with these argsFlat changes.

[matepek]

I can see the problem but I'm missing the rest of the log. Would you attach the full log?

[matepek]

Never mind, I think I found it.

[matepek]

Btw what does your runtest.sh do?

[matepek]

Fixed on next release. Still interested about runtest.sh

[Trass3r]

Thanks!
It sets up the required environment or runs the test via qemu, stuff like that.

[matepek]

Does debug test work for you?

[Trass3r]

For env scripts yes with gdb's set exec-wrapper.
But with qemu it's more complicated.