Test debugging does not work when vs code is attached to docker container

[MichaelMakarov]

Brief Issue Summary

I use vs code for remote development in docker container. c/c++ extensions pack, cmake tools and dev containers are used for this purpose. There is an unit test in cmake project added for ctest. I have tested two scenarious:

1. I have built the project locally on host machine and have succesfully run in debugging mode from test explorer
2. I have built the project in docker container and have failed to debug the test with output "launch property program is missing or empty".
   I have used this launch.json in both cases:
   {
   "version": "0.2.0",
   "configurations": [
   {
   "name": "(ctest) Launch",
   "type": "cppdbg",
   "request": "launch",
   // Resolved by CMake Tools:
   "cwd": "${cmake.testWorkingDirectory}",
   "program": "${cmake.testProgram}",
   "args": [ "${cmake.testArgs}"],
   }
   ]
   }

CMake Tools Diagnostics

{
"os": "linux",
"vscodeVersion": "1.94.2",
"cmtVersion": "1.18.42",
"configurations": [
{
"folder": "/home/docker_bind/projects",
"cmakeVersion": "3.10.2",
"configured": true,
"generator": "Unix Makefiles",
"usesPresets": false,
"compilers": {
"C": "/usr/bin/gcc",
"CXX": "/usr/bin/g++"
}
}
],
"cpptoolsIntegration": {
"isReady": true,
"hasCodeModel": true,
"activeBuildType": "Debug",
"buildTypesSeen": [
"Debug"
],
"requests": [],
"responses": [],
"partialMatches": [],
"targetCount": 12,
"executablesCount": 11,
"librariesCount": 1,
"targets": [
{
"name": "test_cond_var",
"type": "EXECUTABLE"
},
{
"name": "map",
"type": "EXECUTABLE"
},
{
"name": "syscall",
"type": "EXECUTABLE"
},
{
"name": "format_arguments",
"type": "EXECUTABLE"
},
{
"name": "queue_concurrent_performance",
"type": "EXECUTABLE"
},
{
"name": "queue_lib",
"type": "INTERFACE_LIBRARY"
},
{
"name": "queue_consequent_performance",
"type": "EXECUTABLE"
},
{
"name": "queue_concurrent_robustness",
"type": "EXECUTABLE"
},
{
"name": "allocator_test_exe",
"type": "EXECUTABLE"
},
{
"name": "tagged_ptr_128_test",
"type": "EXECUTABLE"
},
{
"name": "comparison_exe",
"type": "EXECUTABLE"
},
{
"name": "address_exe",
"type": "EXECUTABLE"
}
]
},
"settings": [
{
"communicationMode": "automatic",
"useCMakePresets": "auto",
"configureOnOpen": true
}
]
}

Debug Log

2024-11-07T12:42:52.154Z [debug] [expand] expanding cmake
2024-11-07T12:42:52.238Z [info] [proc] Executing command: /usr/bin/cmake --version
2024-11-07T12:42:52.258Z [info] [proc] Executing command: /usr/bin/cmake -E capabilities
2024-11-07T12:42:52.335Z [debug] [main] Safe constructing new CMakeProject instance
2024-11-07T12:42:52.337Z [debug] [variant] Constructing VariantManager
2024-11-07T12:42:52.338Z [debug] [main] Constructing new CMakeProject instance
2024-11-07T12:42:52.339Z [debug] [main] Starting CMake Tools second-phase init
2024-11-07T12:42:52.339Z [debug] [expand] expanding ${workspaceFolder}
2024-11-07T12:42:52.389Z [info] [variant] Loaded new set of variants
2024-11-07T12:42:52.391Z [debug] [kit] Reading kits file /home/docker_bind/.local/share/CMakeTools/cmake-tools-kits.json
2024-11-07T12:42:52.412Z [info] [kit] Successfully loaded 1 kits from /home/docker_bind/.local/share/CMakeTools/cmake-tools-kits.json
2024-11-07T12:42:52.412Z [debug] [expand] expanding /usr/bin/gcc
2024-11-07T12:42:52.413Z [debug] [expand] expanding /usr/bin/g++
2024-11-07T12:42:52.413Z [debug] [expand] expanding cmake
2024-11-07T12:42:52.418Z [debug] [kit] Not reading non-existent kits file: /home/docker_bind/projects/.vscode/cmake-kits.json
2024-11-07T12:42:52.419Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:42:52.421Z [debug] [main] Injecting new Kit into CMake driver
2024-11-07T12:42:52.476Z [debug] [main] CMakeProject instance initialization complete.
2024-11-07T12:42:52.614Z [debug] [extension] Configuring workspace on open /home/docker_bind/projects
2024-11-07T12:42:52.615Z [debug] [expand] expanding cmake
2024-11-07T12:42:52.719Z [debug] [main] Starting new CMake driver
2024-11-07T12:42:52.719Z [debug] [main] Starting CMake driver
2024-11-07T12:42:52.719Z [debug] [main] Using generator from user configuration: Unix Makefiles
2024-11-07T12:42:52.720Z [debug] [driver] Initializing base driver using kit
2024-11-07T12:42:52.720Z [debug] [kit] Testing GCC binary: /usr/bin/gcc
2024-11-07T12:42:52.720Z [info] [proc] Executing command: /usr/bin/gcc -v
2024-11-07T12:42:52.727Z [debug] [kit] Detected GCC compiler: /usr/bin/gcc
2024-11-07T12:42:52.727Z [debug] [driver] CMakeDriver Kit set to GCC 7.5.0 x86_64-linux-gnu
2024-11-07T12:42:52.728Z [debug] [kit] The environment for kit 'GCC 7.5.0 x86_64-linux-gnu': {
"USER": "bamboo",
"HOSTNAME": "usrp-build",
"HOME": "/home/docker_bind",
"container": "podman",
"TERM": "xterm",
"PATH": "/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy/bin/remote-cli:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"VSCODE_AGENT_FOLDER": "/home/docker_bind/.vscode-server",
"DEBIAN_FRONTEND": "noninteractive",
"SHELL": "/bin/bash",
"PWD": "/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy",
"TZ": "Europe/Moscow",
"VSCODE_CWD": "/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy",
"VSCODE_NLS_CONFIG": "{"userLocale":"en","osLocale":"en","resolvedLanguage":"en","defaultMessagesFile":"/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy/out/nls.messages.json","locale":"en","availableLanguages":{}}",
"VSCODE_HANDLES_SIGPIPE": "true",
"VSCODE_AMD_ENTRYPOINT": "vs/workbench/api/node/extensionHostProcess",
"VSCODE_HANDLES_UNCAUGHT_ERRORS": "true",
"REMOTE_CONTAINERS_SOCKETS": "["/tmp/vscode-ssh-auth-d654e4a9-5a4e-4173-b22a-1b90b6fef287.sock","/tmp/.X11-unix/X21","/home/docker_bind/.gnupg/S.gpg-agent"]",
"REMOTE_CONTAINERS_IPC": "/tmp/vscode-remote-containers-ipc-d654e4a9-5a4e-4173-b22a-1b90b6fef287.sock",
"SHLVL": "1",
"": "/bin/cat",
"SSH_AUTH_SOCK": "/tmp/vscode-ssh-auth-d654e4a9-5a4e-4173-b22a-1b90b6fef287.sock",
"DISPLAY": ":21",
"REMOTE_CONTAINERS_DISPLAY_SOCK": "/tmp/.X11-unix/X21",
"REMOTE_CONTAINERS": "true",
"BROWSER": "/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy/bin/helpers/browser.sh",
"ELECTRON_RUN_AS_NODE": "1",
"VSCODE_IPC_HOOK_CLI": "/tmp/vscode-ipc-18b5dcba-e058-44d3-bcf3-7f72a832c9e7.sock",
"VSCODE_L10N_BUNDLE_LOCATION": "",
"APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL": "1"
}
2024-11-07T12:42:52.728Z [debug] [cms-driver] Run doRefreshExpansions
2024-11-07T12:42:52.729Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:42:52.730Z [debug] [expand] expanding ${workspaceFolder}/build
2024-11-07T12:42:52.735Z [debug] [cms-client] Started new CMake Server instance with PID 111955
2024-11-07T12:42:53.738Z [debug] [cache] Reading CMake cache file /home/docker_bind/projects/build/CMakeCache.txt
2024-11-07T12:42:53.742Z [debug] [cache] Parsing CMake cache string
2024-11-07T12:42:53.751Z [debug] [driver] Setting new variant Debug
2024-11-07T12:42:53.751Z [debug] [cms-driver] Run doRefreshExpansions
2024-11-07T12:42:53.751Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:42:53.752Z [debug] [expand] expanding ${workspaceFolder}/build
2024-11-07T12:42:53.761Z [debug] [cms-client] Started new CMake Server instance with PID 111974
2024-11-07T12:42:54.766Z [debug] [cache] Reading CMake cache file /home/docker_bind/projects/build/CMakeCache.txt
2024-11-07T12:42:54.771Z [debug] [cache] Parsing CMake cache string
2024-11-07T12:42:54.788Z [info] [main] Configuring project: projects
2024-11-07T12:42:54.788Z [debug] [main] Saving open files before configure/build
2024-11-07T12:42:54.840Z [debug] [cms-driver] Run doRefreshExpansions
2024-11-07T12:42:54.840Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:42:54.841Z [debug] [expand] expanding ${workspaceFolder}/build
2024-11-07T12:42:54.842Z [debug] [driver] Start configure
2024-11-07T12:42:54.842Z [debug] [driver] Running pre-configure checks and steps
2024-11-07T12:42:54.843Z [debug] [driver] Using compilers in GCC 7.5.0 x86_64-linux-gnu for configure
2024-11-07T12:42:54.843Z [debug] [expand] expanding -DCMAKE_EXPORT_COMPILE_COMMANDS=1
2024-11-07T12:42:54.844Z [debug] [expand] expanding --no-warn-unused-cli
2024-11-07T12:42:54.844Z [debug] [expand] expanding -DCMAKE_BUILD_TYPE:STRING=Debug
2024-11-07T12:42:54.844Z [debug] [expand] expanding -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE
2024-11-07T12:42:54.844Z [debug] [expand] expanding -DCMAKE_C_COMPILER:FILEPATH=/usr/bin/gcc
2024-11-07T12:42:54.844Z [debug] [expand] expanding -DCMAKE_CXX_COMPILER:FILEPATH=/usr/bin/g++
2024-11-07T12:42:54.844Z [debug] [cms-driver] Run doRefreshExpansions
2024-11-07T12:42:54.845Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:42:54.845Z [debug] [expand] expanding ${workspaceFolder}/build
2024-11-07T12:42:54.852Z [info] [cmake] Configuring done
2024-11-07T12:42:54.864Z [info] [cmake] Generating done
2024-11-07T12:42:55.129Z [debug] [expand] expanding cmake
2024-11-07T12:42:55.209Z [debug] [expand] expanding cmake
2024-11-07T12:42:55.212Z [debug] [expand] expanding cmake
2024-11-07T12:42:55.231Z [debug] [expand] expanding -T
2024-11-07T12:42:55.232Z [debug] [expand] expanding test
2024-11-07T12:42:55.232Z [debug] [expand] expanding --output-on-failure
2024-11-07T12:42:55.342Z [debug] [cache] Reading CMake cache file /home/docker_bind/projects/build/CMakeCache.txt
2024-11-07T12:42:55.342Z [debug] [cache] Parsing CMake cache string
2024-11-07T12:42:55.347Z [debug] [expand] expanding cmake
2024-11-07T12:43:00.208Z [debug] [expand] expanding cmake
2024-11-07T12:43:09.814Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:43:09.821Z [info] [main] Building folder: projects
2024-11-07T12:43:09.821Z [debug] [expand] expanding cmake
2024-11-07T12:43:09.912Z [debug] [main] Saving open files before configure/build
2024-11-07T12:43:09.917Z [debug] [expand] expanding cmake
2024-11-07T12:43:09.970Z [debug] [expand] expanding cmake
2024-11-07T12:43:10.014Z [debug] [expand] expanding cmake
2024-11-07T12:43:10.020Z [info] [build] Starting build
2024-11-07T12:43:10.031Z [debug] [driver] Start build all
2024-11-07T12:43:10.034Z [debug] [expand] expanding --build
2024-11-07T12:43:10.034Z [debug] [expand] expanding /home/docker_bind/projects/build
2024-11-07T12:43:10.035Z [debug] [expand] expanding --config
2024-11-07T12:43:10.035Z [debug] [expand] expanding Debug
2024-11-07T12:43:10.035Z [debug] [expand] expanding --target
2024-11-07T12:43:10.036Z [debug] [expand] expanding all
2024-11-07T12:43:10.036Z [debug] [expand] expanding --
2024-11-07T12:43:10.036Z [debug] [expand] expanding -j
2024-11-07T12:43:10.037Z [debug] [expand] expanding 14
2024-11-07T12:43:10.039Z [info] [proc] Executing command: /usr/bin/cmake --build /home/docker_bind/projects/build --config Debug --target all -- -j 14
2024-11-07T12:43:10.039Z [debug] [proc] with environment: {"USER":"bamboo","HOSTNAME":"usrp-build","HOME":"/home/docker_bind","container":"podman","TERM":"xterm","PATH":"/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy/bin/remote-cli:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","VSCODE_AGENT_FOLDER":"/home/docker_bind/.vscode-server","DEBIAN_FRONTEND":"noninteractive","SHELL":"/bin/bash","PWD":"/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy","TZ":"Europe/Moscow","VSCODE_CWD":"/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy","VSCODE_NLS_CONFIG":"{"userLocale":"en","osLocale":"en","resolvedLanguage":"en","defaultMessagesFile":"/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy/out/nls.messages.json","locale":"en","availableLanguages":{}}","VSCODE_HANDLES_SIGPIPE":"true","VSCODE_AMD_ENTRYPOINT":"vs/workbench/api/node/extensionHostProcess","VSCODE_HANDLES_UNCAUGHT_ERRORS":"true","REMOTE_CONTAINERS_SOCKETS":"["/tmp/vscode-ssh-auth-d654e4a9-5a4e-4173-b22a-1b90b6fef287.sock","/tmp/.X11-unix/X21","/home/docker_bind/.gnupg/S.gpg-agent"]","REMOTE_CONTAINERS_IPC":"/tmp/vscode-remote-containers-ipc-d654e4a9-5a4e-4173-b22a-1b90b6fef287.sock","SHLVL":"1","":"/bin/cat","SSH_AUTH_SOCK":"/tmp/vscode-ssh-auth-d654e4a9-5a4e-4173-b22a-1b90b6fef287.sock","DISPLAY":":21","REMOTE_CONTAINERS_DISPLAY_SOCK":"/tmp/.X11-unix/X21","REMOTE_CONTAINERS":"true","BROWSER":"/home/docker_bind/.vscode-server/bin/384ff7382de624fb94dbaf6da11977bba1ecd427-legacy/bin/helpers/browser.sh","ELECTRON_RUN_AS_NODE":"1","VSCODE_IPC_HOOK_CLI":"/tmp/vscode-ipc-18b5dcba-e058-44d3-bcf3-7f72a832c9e7.sock","VSCODE_L10N_BUNDLE_LOCATION":"","APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL":"1","NINJA_STATUS":"[%s/%t %p :: %e] "}
2024-11-07T12:43:10.101Z [info] [build] [ 8%] Built target format_arguments
2024-11-07T12:43:10.102Z [info] [build] [ 16%] Built target comparison_exe
2024-11-07T12:43:10.103Z [info] [build] [ 25%] Built target address_exe
2024-11-07T12:43:10.103Z [info] [build] [ 33%] Built target queue_consequent_performance
2024-11-07T12:43:10.104Z [info] [build] [ 41%] Built target test_cond_var
2024-11-07T12:43:10.104Z [info] [build] [ 54%] Built target map
2024-11-07T12:43:10.106Z [info] [build] [ 79%] Built target queue_concurrent_performance
2024-11-07T12:43:10.106Z [info] [build] [ 79%] Built target queue_concurrent_robustness
2024-11-07T12:43:10.106Z [info] [build] [ 79%] Built target syscall
2024-11-07T12:43:10.107Z [info] [build] [100%] Built target allocator_test_exe
2024-11-07T12:43:10.108Z [info] [build] [100%] Built target tagged_ptr_128_test
2024-11-07T12:43:10.122Z [info] [driver] Build completed: 00:00:00.091
2024-11-07T12:43:10.128Z [debug] [expand] expanding cmake
2024-11-07T12:43:10.129Z [debug] [cms-driver] Run doRefreshExpansions
2024-11-07T12:43:10.129Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:43:10.130Z [debug] [expand] expanding ${workspaceFolder}/build
2024-11-07T12:43:10.175Z [info] [build] Build finished with exit code 0
2024-11-07T12:43:10.177Z [debug] [expand] expanding cmake
2024-11-07T12:43:10.177Z [debug] [cache] Reading CMake cache file /home/docker_bind/projects/build/CMakeCache.txt
2024-11-07T12:43:10.179Z [debug] [cache] Parsing CMake cache string
2024-11-07T12:43:10.202Z [debug] [expand] expanding cmake
2024-11-07T12:43:10.213Z [debug] [expand] expanding /home/docker_bind/projects
2024-11-07T12:43:20.714Z [debug] [extension] [7147] cmake.viewLog started
2024-11-07T12:43:20.763Z [debug] [extension] [7147] cmake.viewLog finished (returned undefined)

Additional Information

[Amy-Li03]

Hi @MichaelMakarov , thanks for your feedback!
Is there any way that you could provide a minimal repro project so that we can investigate?

[MichaelMakarov]

There is a simple cmake project with one test. Using it one can achieve the behaviour I have described before.example.zip

[Amy-Li03]

@MichaelMakarov thanks for your quick response!
We try to debug above example project on host Linux machine and Linux machine with docker container, however they all debug failed with below message. Please correct us if we have done something wrong.

[MichaelMakarov]

I have performed the following steps on my host machine:

1. build the project
2. create the breakpoint in generator.t.cpp:14
3. open the testing tab (there must be generator test as shown in the first image)
4. click debug test (the debugging must stop on the breakpoint as shown in the second image)

I have performed the following steps in remote container case:

1. use dev container extension and attach to running container (I use podman for building and running containers)
2. open folder with the project
3. perform the steps 1 - 4 as described above in host machine section

[Amy-Li03]

Hi @MichaelMakarov , thanks for your detailed repro steps.
Firstly, I performed the steps on a Linux host machine, it works as expected: the debugging stops on the breakpoint after clicking debug test from test explorer.
Then, I performed the steps in remote container case, use dev container extension and docker to attach to running container, it also works as on host machine, please see following screenshot and video:

Attached my test project: vs-remote-test.zip, it contains devcontainer.json and Dockerfile.

Remote_debugTest.webm

[MichaelMakarov]

In my opinion, the main difference is that I have prepared docker container outside vs code environment and started it via terminal. Then I used dev containers extension to attach the running docker container and open the project directory inside it. I anticipate that vs code prepares some additional info in your case but don't know what it actually is.

[Amy-Li03]

Do you get above results I shared when using a docker container inside vs code environment?

[MichaelMakarov]

Yes I do. I think so. I have reproduced the following steps to achieve your result:

1. click new dev container button in dev containers extension
2. select dev container c++
3. copy the project to the new docker volume that is created previously
4. build and launch debugging.

Is it correct?

[Amy-Li03]

To summarize, this issue reproduced when using docker container outside vs code environment and started it via terminal, however, it works well when using a docker container inside vs code environment.

@gcampbell-msft could you please confirm whether this is issue on our side?

[gcampbell-msft]

Interesting, I wonder if this is something to do with path differences?

@bwateratmsft, I see you work on the docker extension, I wonder if you have any insight?

[bwateratmsft]

Hm, this sounds like a tricky one. I think the best thing would be to debug the CMake tools' debugging code, and see how it is resolving that value for ${cmake.testProgram}. In containers started by VSCode it seems to be working as expected but in containers started by the terminal it is not.

The first thing I would look at is environment variables--there may be subtle differences in environment, especially PATH, that are causing this. The next would be to look at VSCode user, workspace, and remote settings. There is a command for each of those, "Preferences: Open User|Workspace|Remote Settings", preferably JSON because that's easier to compare quickly.