[Bug]: Headers present in the sysroot but not available in the bazel sandbox

[0x2Adr1]

What happened?

Some headers that are definitely present in the default toolchain's sysroot are not copied in the bazel sandbox.

For example bits/types/struct___jmp_buf_tag.h.

See https://gist.github.com/0x2Adr1/51635421f64d6de8cbb14dbf40a27a4e for a minimal code that reproduces the issue.

  exec env - \
    BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \
    PATH=/bin:/usr/bin:/usr/local/bin \
    PWD=/proc/self/cwd \
  external/gcc_toolchain_x86_64/bin/gcc -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/k8-fastbuild/bin/_objs/main/main.pic.d '-frandom-seed=bazel-out/k8-fastbuild/bin/_objs/main/main.pic.o' -fPIC '-DBAZEL_CURRENT_REPOSITORY=""' -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools --sysroot external/sysroot_x86_64 -no-canonical-prefixes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-std=c++17' -isystemexternal/sysroot_x86_64//include/c++/10.3.0 -isystemexternal/sysroot_x86_64//include/c++/10.3.0/x86_64-linux -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include-fixed -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include -isystemexternal/sysroot_x86_64//usr/include -c main.cpp -o bazel-out/k8-fastbuild/bin/_objs/main/main.pic.o)
# Configuration: f68de507e19f0236b7b3947a923f4eddf8a133099bf7d87ad0db3cf41a440b09
# Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
main.cpp:3:10: fatal error: bits/types/struct___jmp_buf_tag.h: No such file or directory
    3 | #include <bits/types/struct___jmp_buf_tag.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Target //:main failed to build
INFO: Elapsed time: 0.798s, Critical Path: 0.13s
INFO: 4 processes: 4 internal.
FAILED: Build did NOT complete successfully

But this header is definitely present if I download and look at the toolchain content manually:

$ ls x86-64--glibc--stable-2021.11-5/x86_64-buildroot-linux-gnu/sysroot/usr/include/bits/types/struct___jmp_buf_tag.h 
x86-64--glibc--stable-2021.11-5/x86_64-buildroot-linux-gnu/sysroot/usr/include/bits/types/struct___jmp_buf_tag.h

Version

Development (host) and target OS/architectures: ArchLinux x86_64

Output of bazel --version: bazel 6.0.0

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file: 22862ac

Language(s) and/or frameworks involved: C++

How to reproduce

Minimal repro code: https://gist.github.com/0x2Adr1/51635421f64d6de8cbb14dbf40a27a4e

Any other information?

No response

Fund our work

• Sponsor our open source work by donating a bug bounty

[kriswuollett]

As mentioned in #134, I've encountered same issue here for stddef.h which should be in "-isystemexternal/sysroot_aarch64//lib/gcc/aarch64-linux/10.3.0/include". When I patch the gcc-toolchain repo to also include /usr/include/linux I see it in the invocation, "-isystemexternal/sysroot_aarch64//usr/include/linux" but still can't find it:

running: "/home/kris/.cache/bazel/_bazel_kris/ebd20a16b5bb6af41a7a481997cc3984/sandbox/linux-sandbox/1077/execroot/my-code/external/gcc_toolchain_aarch64/bin/gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "--sysroot" "external/sysroot_aarch64" "-no-canonical-prefixes" "-fno-canonical-system-headers" "-Wno-builtin-macro-redefined" "-D__DATE__=\"redacted\"" "-D__TIMESTAMP__=\"redacted\"" "-D__TIME__=\"redacted\"" "-fdiagnostics-color=always" "-nostdinc" "-Bexternal/gcc_toolchain_aarch64/bin" "-isystemexternal/sysroot_aarch64//aarch64-linux/include/c++/10.3.0" "-isystemexternal/sysroot_aarch64//aarch64-linux/include/c++/10.3.0/aarch64-linux" "-isystemexternal/sysroot_aarch64//lib/gcc/aarch64-linux/10.3.0/include-fixed" "-isystemexternal/sysroot_aarch64//lib/gcc/aarch64-linux/10.3.0/include" "-isystemexternal/sysroot_aarch64//usr/include" "-isystemexternal/sysroot_aarch64//usr/include/linux" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-fvisibility=hidden" "-ffunction-sections" "-fdata-sections" "-fmerge-all-constants" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-o" "/home/kris/.cache/bazel/_bazel_kris/ebd20a16b5bb6af41a7a481997cc3984/sandbox/linux-sandbox/1077/execroot/my-code/bazel-out/k8-fastbuild-ST-87f845362927/bin/external/crate_index__zstd-sys-2.0.7-zstd.1.5.4/zstd-sys_build_script.out_dir/zstd/lib/common/error_private.o" "-c" "zstd/lib/common/error_private.c"
cargo:warning=In file included from zstd/lib/common/error_private.h:24,
cargo:warning=                 from zstd/lib/common/error_private.c:13:
cargo:warning=zstd/lib/common/../zstd_errors.h:19:10: fatal error: stddef.h: No such file or directory
cargo:warning=   19 | #include <stddef.h>   /* size_t */
cargo:warning=      |          ^~~~~~~~~~
cargo:warning=compilation terminated.
exit status: 1