How to run command before tests?

[dkuschmierz]

Describe the solution you'd like
I would like to have an option where I can set a command which is run before any test is run (before scanning, running, debugging etc.) Background is, that we set up our build environment with conan and I cannot run the tests if the build-environment is not set up properly. Therefore I currently have to start vscode via console and run the command before i run vscode.

[matepek]

Users are usually creating a .bat/.sh wrapper for their executables.
If you are using a decent build tool, you can do it easily. For example in case of cmake: see. (edited)

My problem is that what kind of command are you thinking about? Linux, mac, windows? all?
You see hard to give a general solution. What I think I could do to call a vscode task but you might get the same result configuring a task with

"runOptions": {
        "runOn": "folderOpen"
},

Here is what I am using for similar purpose:

"tasks": [
    {
      "label": "createTmpWorkingFolder",
      "type": "shell",
      "command": "mkdir -p out/tmp/workspaceFolder",
      "windows": { "command": "mkdir out\\tmp\\workspaceFolder" }
    },
    {
      "type": "npm",
      "script": "watch",
      "problemMatcher": "$tsc-watch",
      "isBackground": true,
      "presentation": {
        "reveal": "never"
      },
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "runOptions": {
        "runOn": "folderOpen"
      },
      "dependsOn": ["createTmpWorkingFolder"]
    }
  ]

Note:

• second task dependsOn first task
• first task executes a different command on windows

See vscode task documentation for further details

[dkuschmierz]

Yes we also have a activation-script. For my other tasks I run that script before running e.g. the build command. But I don't have tasks for running/scanning the tests.

My opinion is, that before running Mytest.exe another command should be called. In my case activate.bat from conan. Like a prerun command

[matepek]

like catch2TestExplorer.dependsOnTask: <taskname> where is a task defined in tasks.json.
That would be a nice solution don't you think?

[dkuschmierz]

Yes, that would be great!

[matepek]

You have to run it always before you run your exec, right?
What does actually do?
When do you have to rerun it?

Example: there is exec1 and exec2.

Test runs in batches, so is it ok to run the task and then run exec1 and then exec2?

[dkuschmierz]

It's only necessary once after opening the batches. So opening a batch, run the command once and afterwards run all tests would be fine.

The script basically modifies the PATH variable and adds the necessary toolchain-folders temporarily

[matepek]

What is a batch? Not fully understand. Consecutive runs don't need to rerun the task?

FYI, you can modify the PATH environment variables.
with env: "env": { "PATH": "${os_env:PATH};<custom-path>" }
See details.

[dkuschmierz]

Ah, I think I misunderstood "batch", I thought of a command bash.
What we do on windows is:

1. Open cmd
2. Call activate script
3. Run-Unit tests

I've seen the "env" option, but as the paths are changing, I cannot set a static value in the env-option

[matepek]

Ah, in this case I think the catch2TestExplorer.dependsOnTask proposal wouldn't solve your problem.

Env modification is only available in the current process and it's children.

what you can do:

• write a startvscode.bat like:

init.bat
code sourcefolder

or

• generate executable wrappers as I mentioned previously.

I could develop this feature, but it wouldn't work for your scenario anyway.

[vakokako]

Upvoting this issue. There is a need to source some dynamic environment variables, which cannot be done with "env": { ... }. So the ability to run a custom command before the executables run would be great!)

[matepek]

Getting the environment variables from the custom program would be problematic though.

It is not impossible, but the whole concept is more "inheritance" like.
Please read my comments on this issue, also : https://en.wikipedia.org/wiki/Environment_variable

My conclusion: running a custom task won't be able to set the environment variables of the test executable --> Generate a wrapper.

[dkuschmierz]

But wouldn‘t it be possible to run the test in a shell with:

1. Open Shell
2. Run custom script
3. Run test
4. Parse output

[matepek]

Linux script? Mac script? (I know.. I meant the os specific programs) Windows script?

You see, I could give a not so general solution. Instead this everybody can write their own wrapper scripts. More effective to generate those scripts. For example: in case of cmake check add_custom_command

I guess I should write an example. Thumbs up if you would read it.

[xahon]

Why this issue is closed? I couldn't find a solution to rebuild project before testing

[matepek]

And you won't @xahon,

This extension support the other way around. You rebuild your project and the extension detects the change and runs your 'changed' executables if autorun is set:

Anyway this issue is about running some custom command before the test.

[matepek]

Hey @dkuschmierz and @friendnick ,

I created a cmake wrapper generator example.
You can check it here.

[dkuschmierz]

Now I understand what you ment. Works perfectly, thanks a lot!

[matepek]

Happy to hear that.

Also, don't forget to replace catch2TestExplorer.executables pattern to a stricter one like "{build,Build,BUILD,out,Out,OUT}/**/*.test-wrapper*" because we don't want to match the executables just the wrappers.

[dkuschmierz]

Hi @matepek
any idea how to set up the debug configuration with this wrapper?

[matepek]

Hello @dkuschmierz ,

I'm afraid one cannot use debug feature with a wrapper.

But.. there is another way to get the environment variables and just a slightly advanced. You can check the example here.

[matepek]

Hello @dkuschmierz ,
Just wondering that this second example was clear enough and solved your problem or not. ?

[matepek]

Hello there will be a dependsOnTask under testMate.test.advancedExecutables in the next version (3.0.4)
With that one can run custom vscode command before the tests are run.

BUT it won't solve the environment variable issue: Additional variables under the execution of the task wont be propagated to the test executable.

[matepek]

I decided adding a new feature under testMate.advancedExecutables called envFile.
I think this is interesting for everyone who is reading this issue.
Check here

  "testMate.test.advancedExecutables": [{
    "pattern": "<default pattern>",
    "envFile": "out/env.json",
    "dependsOn": "out/env.json"
  }]

[matepek]

Added advancedExecutables.executionWrapper mostly for emulators.

[umbi-umbi]

kalie dificuldade waffa