Environmens variables are not taken into account when discovering tests

[jackorobot]

Situation

We test the libraries we develop using googletest. One of our library depend on an external (closed source, shipped as .so) library. This library is dynamically linked to the test executable. It requires some environment variables to be set, which are directories relative to the executable. This works when setting these in a shell script, or in terminal. When these environment variables are not set, the execution will end with an return code of 255.

Issue

We want to start using this plugin for automation in vscode, so we tried adding this to our settings.json:

"catch2TestExplorer.executables": [
        {
            "name": "${filename}",
            "description": "${relDirpath}/",
            "pattern": "folder/to/test/executables/*",
            "cwd":"${absDirpath}",
            "env": {
                "VARIABLE_1": "${absDirPath}/../relative/directory",
                "VARIABLE_2": "${absDirPath}/../other/relative/directory"
            }
        }
    ]

But this test executable does not appear in the test explorer, while others appear just fine. Also with logging output as seen below in mind I think that during the test discovery the environment variables do not get set correctly.

I cannot find the console output that happens during the test discovery. Maybe this can give some insight to what is happening.

Logging output:

[2019-08-08 13:49:38.754] [WARN] [at load.suiteCreationAndLoadingTasks.push.c2fs.isNativeExecutableAsync.then._createSuiteByUri.then (/home/me/.vscode/extensions/matepek.vscode-catch2-test-adapter-2.6.6/out/src/TestExecutableInfo.js:73:38)] Not a test executable: /directory/to/test/executable reason: [Error: Not a supported test executable: /directory/to/test/executable
 output: [object Object]
	at determineTestTypeOfExecutable.c2fs.isNativeExecutableAsync.then.c2fs.spawnAsync.then (/home/me/.vscode/extensions/matepek.vscode-catch2-test-adapter-2.6.6/out/src/TestSuiteInfoFactory.js:42:23)]

Version numbers

• Extension Version 2.6.6
• VS Code 1.36.1
• Google Test Version 1.8.1
• Manjaro 18.0.4

[matepek]

Hello,

I'm sorry, but your crafted log not enough to understand whats happening.

What is a bit suspicious is that you are mentioning relative path:

It requires some environment variables to be set, which are directories relative to the executable

But you are actually using absolute paths:

"VARIABLE_1": "${absDirPath}/../relative/directory",
"VARIABLE_2": "${absDirPath}/../other/relative/directory"

Try:

"VARIABLE_1": "../relative/directory",
...

[jackorobot]

Hi,
I understand that it is too short, but I am afraid not more info was available in the log.
You are absolutely right about those paths. Until now I assumed that the method we used in our shell scripts would work in the configs as well.
I will research if all is well, but for now I will close this. Thanks for the quick response!

[matepek]

Here is an alternative approach