Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clang produces error on compiling assembly files when using plugins like polly #88173

Closed
StormBytePP opened this issue Apr 9, 2024 · 8 comments · Fixed by #88948
Closed

Clang produces error on compiling assembly files when using plugins like polly #88173

StormBytePP opened this issue Apr 9, 2024 · 8 comments · Fixed by #88948
Labels
clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' polly

Comments

@StormBytePP
Copy link

StormBytePP commented Apr 9, 2024 •
edited
Loading

Suppose we have a simple test program called test.cxx and we have polly plugin LLVMPolly.so built dynamically along with LLVM (with LLVM_POLLY_LINK_INTO_TOOLS=OFF):

#include <iostream>

int main() {
	std::cout << "Hello world!" << std::endl;
        return 0;
}

Compiling it with clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -o test test.cxx works as expected, but what happens when compiling assembly?

To test it, we create our test assembly test.S with clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -S -c -o test.S test.cxx and then we try to compile test.S as object, then we get:

clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -c -o test.S.o test.S 
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?
clang (LLVM option parsing): Unknown command line argument '-polly-parallel'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--ppc-gen-isel'?

It seems that it is not loading the plugin and not recognizing the options that worked in previous phases (I assume because they are not needed/used when compiling assembly)

I think this is a bug because in this case it is much better to load and ignore plugin options if they are not used rather than throw an error because polly use flags might be present in C{XX}FLAGS making the whole target program to fail to compile if it includes assembly files.

One real example of it is the LLVM library itself which fails to compile when CXXFLAGS with polly options are used on some assembly files that LLVM compiles:

[126/3501 (  3%)] /usr/lib/ccache/bin/clang -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/include -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/include  -DNDEBUG -O2 -pipe -march=native -Wno-unused-command-line-argument -O3 -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -mllvm -polly-vectorizer=stripmine -mllvm -polly-run-dce -mllvm -polly-scheduling-chunksize=1 -mllvm -polly-scheduling=dynamic -mllvm -polly-num-threads=32 -mllvm -polly-omp-backend=LLVM -mllvm -polly-invariant-load-hoisting -fopenmp  -flto=thin -fPIC -MD -MT lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -MF lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o.d -o lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -c /tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_unix.S
FAILED: lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o 
/usr/lib/ccache/bin/clang -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/include -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/include  -DNDEBUG -O2 -pipe -march=native -Wno-unused-command-line-argument -O3 -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -mllvm -polly-vectorizer=stripmine -mllvm -polly-run-dce -mllvm -polly-scheduling-chunksize=1 -mllvm -polly-scheduling=dynamic -mllvm -polly-num-threads=32 -mllvm -polly-omp-backend=LLVM -mllvm -polly-invariant-load-hoisting -fopenmp  -flto=thin -fPIC -MD -MT lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -MF lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o.d -o lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -c /tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_unix.S
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?
clang (LLVM option parsing): Unknown command line argument '-polly-parallel'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--ppc-gen-isel'?
clang (LLVM option parsing): Unknown command line argument '-polly-vectorizer=stripmine'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--slp-vectorize-hor=stripmine'?
clang (LLVM option parsing): Unknown command line argument '-polly-run-dce'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--unroll-runtime'?
clang (LLVM option parsing): Unknown command line argument '-polly-scheduling-chunksize=1'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--cache-line-size=1'?
clang (LLVM option parsing): Unknown command line argument '-polly-scheduling=dynamic'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--post-RA-scheduler=dynamic'?
clang (LLVM option parsing): Unknown command line argument '-polly-num-threads=32'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--xcore-max-threads=32'?
clang (LLVM option parsing): Unknown command line argument '-polly-omp-backend=LLVM'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--pre-RA-sched=LLVM'?
clang (LLVM option parsing): Unknown command line argument '-polly-invariant-load-hoisting'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--licm-control-flow-hoisting'?

For more info look the Gentoo PR I did for trying to package Polly for Gentoo

Edit: It is worth to say that this affects all clang versions including 19 git version.
@github-actions github-actions bot added the clang Clang issues not falling into any other category label Apr 9, 2024
@StormBytePP StormBytePP mentioned this issue Apr 9, 2024
Open
@llvmbot
Copy link
Collaborator

llvmbot commented Apr 9, 2024

@llvm/issue-subscribers-polly

Author: StormBytePP (StormBytePP)

Suppose we have a simple test program called test.cxx and we have polly plugin LLVMPolly.so built dynamically along with LLVM (with `LLVM_POLLY_LINK_INTO_TOOLS=OFF`): 
#include &lt;iostream&gt;

int main() {
	std::cout &lt;&lt; "Hello world!" &lt;&lt; std::endl;
        return 0;
}

Compiling it with clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -o test test.cxx works as expected, but what happens when compiling assembly?

To test it, we create our test assembly test.S with clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -S -c -o test.S test.cxx and then we try to compile test.S as object, then we get:

clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -c -o test.S.o test.S 
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?
clang (LLVM option parsing): Unknown command line argument '-polly-parallel'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--ppc-gen-isel'?

It seems that it is not loading the plugin and not recognizing the options that worked in previous phases (I assume because they are not needed/used when compiling assembly)

I think this is a bug because in this case it is much better to load and ignore plugin rather than throw an error because polly use flag might be present in C{XX}FLAGS making the whole program to fail compile.

One real example of it is the LLVM library itself which when polly flags are used fails to compile with:

[126/3501 (  3%)] /usr/lib/ccache/bin/clang -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/include -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/include  -DNDEBUG -O2 -pipe -march=native -Wno-unused-command-line-argument -O3 -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -mllvm -polly-vectorizer=stripmine -mllvm -polly-run-dce -mllvm -polly-scheduling-chunksize=1 -mllvm -polly-scheduling=dynamic -mllvm -polly-num-threads=32 -mllvm -polly-omp-backend=LLVM -mllvm -polly-invariant-load-hoisting -fopenmp  -flto=thin -fPIC -MD -MT lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -MF lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o.d -o lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -c /tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_unix.S
FAILED: lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o 
/usr/lib/ccache/bin/clang -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/include -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/include  -DNDEBUG -O2 -pipe -march=native -Wno-unused-command-line-argument -O3 -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -mllvm -polly-vectorizer=stripmine -mllvm -polly-run-dce -mllvm -polly-scheduling-chunksize=1 -mllvm -polly-scheduling=dynamic -mllvm -polly-num-threads=32 -mllvm -polly-omp-backend=LLVM -mllvm -polly-invariant-load-hoisting -fopenmp  -flto=thin -fPIC -MD -MT lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -MF lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o.d -o lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -c /tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_unix.S
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?
clang (LLVM option parsing): Unknown command line argument '-polly-parallel'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--ppc-gen-isel'?
clang (LLVM option parsing): Unknown command line argument '-polly-vectorizer=stripmine'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--slp-vectorize-hor=stripmine'?
clang (LLVM option parsing): Unknown command line argument '-polly-run-dce'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--unroll-runtime'?
clang (LLVM option parsing): Unknown command line argument '-polly-scheduling-chunksize=1'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--cache-line-size=1'?
clang (LLVM option parsing): Unknown command line argument '-polly-scheduling=dynamic'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--post-RA-scheduler=dynamic'?
clang (LLVM option parsing): Unknown command line argument '-polly-num-threads=32'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--xcore-max-threads=32'?
clang (LLVM option parsing): Unknown command line argument '-polly-omp-backend=LLVM'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--pre-RA-sched=LLVM'?
clang (LLVM option parsing): Unknown command line argument '-polly-invariant-load-hoisting'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--licm-control-flow-hoisting'?

For more info look the Gentoo PR I did for trying to package Polly for Gentoo

@StormBytePP
Copy link
Author

Note: In a test it is discovered that if plugin is forced linked to LLVM library (with patchelf --add-needed LLVMPolly.so libLLVM.so) then the error is gone so it confirms it is clang which is ignoring plugin loading.

@thesamesam thesamesam changed the title CLang produces error on compiling assembly files when using plugins like polly Clang produces error on compiling assembly files when using plugins like polly Apr 10, 2024
@Meinersbur
Copy link
Member

Meinersbur commented Apr 16, 2024 •
edited
Loading

-fplugin is processed by clang -fc1, so when no C/C++/Obj-C files are involved it indeed is not used. From that point of view, the make tool should not have added the -fplugin option. In CMake, this assembler flags are controlled by CMAKE_ASM_FLAGS. For LLVM, you mention CXXFLAGS, but what was your cmake line to reproduce?

However, there are a lot of other flags that are ignored without warning even when not compiling C/C++/Obj-C (e.g. compile flags when only linking) because not all build tools care, and we could just add -fplugin to these.

@StormBytePP
Copy link
Author

StormBytePP commented Apr 16, 2024 •
edited
Loading

-fplugin is processed by clang -fc1, so when no C/C++/Obj-C files are involved it indeed is not used. From that point of view, the make tool should not have added the -fplugin option. In CMake, this assembler flags are controlled by CMAKE_ASM_FLAGS. For LLVM, you mention CXXFLAGS, but what was your cmake line to reproduce?

I'm using Gentoo build system, in concrete (as an example) I can see this line:

cmake -C /tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/gentoo_common_config.cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr -DLLVM_APPEND_VC_REV=OFF -DCMAKE_INSTALL_PREFIX=/usr/lib/llvm/18 -DLLVM_LIBDIR_SUFFIX= -DBUILD_SHARED_LIBS=OFF -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON -DLLVM_DISTRIBUTION_COMPONENTS=LLVM;LTO;Remarks;llvm-config;cmake-exports;llvm-headers;LLVMDemangle;LLVMSupport;LLVMTableGen;llvm_gtest;llvm_gtest_main;LLVMTestingAnnotations;LLVMTestingSupport;LLVMDebuginfod;llvm-tblgen;FileCheck;llvm-PerfectShuffle;count;not;yaml-bench;UnicodeNameMappingGenerator;bugpoint;dsymutil;llc;lli;lli-child-target;llvm-addr2line;llvm-ar;llvm-as;llvm-bcanalyzer;llvm-bitcode-strip;llvm-c-test;llvm-cat;llvm-cfi-verify;llvm-config;llvm-cov;llvm-cvtres;llvm-cxxdump;llvm-cxxfilt;llvm-cxxmap;llvm-debuginfo-analyzer;llvm-debuginfod-find;llvm-diff;llvm-dis;llvm-dlltool;llvm-dwarfdump;llvm-dwarfutil;llvm-dwp;llvm-exegesis;llvm-extract;llvm-gsymutil;llvm-ifs;llvm-install-name-tool;llvm-jitlink;llvm-jitlink-executor;llvm-lib;llvm-libtool-darwin;llvm-link;llvm-lipo;llvm-lto;llvm-lto2;llvm-mc;llvm-mca;llvm-ml;llvm-modextract;llvm-mt;llvm-nm;llvm-objcopy;llvm-objdump;llvm-opt-report;llvm-otool;llvm-pdbutil;llvm-profdata;llvm-profgen;llvm-ranlib;llvm-rc;llvm-readelf;llvm-readobj;llvm-readtapi;llvm-reduce;llvm-remarkutil;llvm-rtdyld;llvm-sim;llvm-size;llvm-split;llvm-stress;llvm-strings;llvm-strip;llvm-symbolizer;llvm-tli-checker;llvm-undname;llvm-windres;llvm-xray;obj2yaml;opt;sancov;sanstats;split-file;verify-uselistorder;yaml2obj;opt-viewer;LLVMgold; -DLLVM_TARGETS_TO_BUILD= -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=X86;NVPTX;AArch64;AMDGPU;ARM;AVR;BPF;Hexagon;Lanai;LoongArch;MSP430;Mips;PowerPC;RISCV;Sparc;SystemZ;VE;WebAssembly;XCore -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_TESTS=ON -DLLVM_BUILD_TESTS=no -DLLVM_INSTALL_GTEST=ON -DLLVM_ENABLE_FFI=yes -DLLVM_ENABLE_LIBEDIT=no -DLLVM_ENABLE_TERMINFO=yes -DLLVM_ENABLE_LIBXML2=yes -DLLVM_ENABLE_ASSERTIONS=no -DLLVM_ENABLE_LIBPFM=no -DLLVM_ENABLE_EH=ON -DLLVM_ENABLE_RTTI=ON -DLLVM_ENABLE_Z3_SOLVER=no -DLLVM_ENABLE_ZLIB=FORCE_ON -DLLVM_ENABLE_ZSTD=FORCE_ON -DLLVM_ENABLE_CURL=no -DLLVM_ENABLE_HTTPLIB=no -DLLVM_HOST_TRIPLE=x86_64-pc-linux-gnu -DFFI_INCLUDE_DIR=/usr/lib/libffi/include -DFFI_LIBRARY_DIR= -DPython3_EXECUTABLE=/usr/bin/python3.12 -DOCAMLFIND=NO -DLLVM_VERSION_SUFFIX= -DLLVM_BUILD_DOCS=OFF -DLLVM_ENABLE_OCAMLDOC=OFF -DLLVM_ENABLE_SPHINX=OFF -DLLVM_ENABLE_DOXYGEN=OFF -DLLVM_INSTALL_UTILS=ON -DLLVM_BINUTILS_INCDIR=/usr/include -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_TOOLCHAIN_FILE=/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/gentoo_toolchain.cmake /tmp/portage/sys-devel/llvm-18.1.3/work/llvm

With toolchain file added by emerge:

set(CMAKE_ASM_COMPILER "clang")
set(CMAKE_ASM-ATT_COMPILER "clang")
set(CMAKE_C_COMPILER "clang")
set(CMAKE_CXX_COMPILER "clang++")
set(CMAKE_Fortran_COMPILER "x86_64-pc-linux-gnu-gfortran")
set(CMAKE_AR /usr/lib/llvm/18/bin/llvm-ar CACHE FILEPATH "Archive manager" FORCE)
set(CMAKE_RANLIB /usr/lib/llvm/18/bin/llvm-ranlib CACHE FILEPATH "Archive index generator" FORCE)
set(CMAKE_SYSTEM_PROCESSOR "x86_64")

The CXXFLAGS I think are set in environment

However, there are a lot of other flags that are ignored without warning even when not compiling C/C++/Obj-C (e.g. compile flags when only linking) because not all build tools care, and we could just add -fplugin to these.

I would thank this move, but not fplugin since it is not the cause as it seems already ignored, what should be ignored are the pairs -mllvm -XXXXX (because they are not recognized if plugin is not loaded) or at least all the polly options so it is not used but it gives no error neither.

Other option would be not to ignore fplugin and let it load, so all plugin options just do not need to be ignored and will not fail even if they are unused, but I don't know if it might cause any undesireable side effect.

Meinersbur added a commit to Meinersbur/llvm-project that referenced this issue Apr 16, 2024
@Meinersbur
Do not warn about unused plugin flags.
e5b10d3 
Plugins are not loaded without the -cc1 phase. Do not report them when
running on a assembly file or when linking. Many build tools add these
options to all driver invocations. See GH llvm#88173.
@Meinersbur Meinersbur mentioned this issue Apr 16, 2024
Merged
@Meinersbur
Copy link
Member

I created #88948.

Note that for loading Polly, you should use -fpass-plugin. -fplugin would have worked with the legacy pass manager, but I think has no effect with the NPM.

@StormBytePP
Copy link
Author

I created #88948.

Note that for loading Polly, you should use -fpass-plugin. -fplugin would have worked with the legacy pass manager, but I think has no effect with the NPM.

Maybe it is a good hint to update polly doc, at least at their website

Meinersbur added a commit to Meinersbur/llvm-project that referenced this issue Apr 30, 2024
@Meinersbur
Do not warn about unused plugin flags.
d2cd620 
Plugins are not loaded without the -cc1 phase. Do not report them when
running on a assembly file or when linking. Many build tools add these
options to all driver invocations. See GH llvm#88173.
Meinersbur added a commit that referenced this issue May 6, 2024
@Meinersbur
[driver] Do not warn about unused plugin flags. (#88948)
6e31a49 
Plugins are not loaded without the -cc1 phase. Do not report them when
running on an assembly file or when linking. Many build tools add these
options to all driver invocations, including LLVM's build system.

Fixes #88173
@EugeneZelenko EugeneZelenko added clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' and removed clang Clang issues not falling into any other category labels May 6, 2024
@llvmbot
Copy link
Collaborator

llvmbot commented May 6, 2024

@llvm/issue-subscribers-clang-driver

Author: StormBytePP (StormBytePP)

Suppose we have a simple test program called test.cxx and we have polly plugin LLVMPolly.so built dynamically along with LLVM (with `LLVM_POLLY_LINK_INTO_TOOLS=OFF`): 
#include &lt;iostream&gt;

int main() {
	std::cout &lt;&lt; "Hello world!" &lt;&lt; std::endl;
        return 0;
}

Compiling it with clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -o test test.cxx works as expected, but what happens when compiling assembly?

To test it, we create our test assembly test.S with clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -S -c -o test.S test.cxx and then we try to compile test.S as object, then we get:

clang++ -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -c -o test.S.o test.S 
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?
clang (LLVM option parsing): Unknown command line argument '-polly-parallel'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--ppc-gen-isel'?

It seems that it is not loading the plugin and not recognizing the options that worked in previous phases (I assume because they are not needed/used when compiling assembly)

I think this is a bug because in this case it is much better to load and ignore plugin options if they are not used rather than throw an error because polly use flags might be present in C{XX}FLAGS making the whole target program to fail to compile if it includes assembly files.

One real example of it is the LLVM library itself which fails to compile when CXXFLAGS with polly options are used on some assembly files that LLVM compiles:

[126/3501 (  3%)] /usr/lib/ccache/bin/clang -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/include -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/include  -DNDEBUG -O2 -pipe -march=native -Wno-unused-command-line-argument -O3 -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -mllvm -polly-vectorizer=stripmine -mllvm -polly-run-dce -mllvm -polly-scheduling-chunksize=1 -mllvm -polly-scheduling=dynamic -mllvm -polly-num-threads=32 -mllvm -polly-omp-backend=LLVM -mllvm -polly-invariant-load-hoisting -fopenmp  -flto=thin -fPIC -MD -MT lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -MF lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o.d -o lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -c /tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_unix.S
FAILED: lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o 
/usr/lib/ccache/bin/clang -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3 -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm_build-abi_x86_64.amd64/include -I/tmp/portage/sys-devel/llvm-18.1.3/work/llvm/include  -DNDEBUG -O2 -pipe -march=native -Wno-unused-command-line-argument -O3 -fplugin=LLVMPolly.so -mllvm -polly -mllvm -polly-parallel -mllvm -polly-vectorizer=stripmine -mllvm -polly-run-dce -mllvm -polly-scheduling-chunksize=1 -mllvm -polly-scheduling=dynamic -mllvm -polly-num-threads=32 -mllvm -polly-omp-backend=LLVM -mllvm -polly-invariant-load-hoisting -fopenmp  -flto=thin -fPIC -MD -MT lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -MF lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o.d -o lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_sse2_x86-64_unix.S.o -c /tmp/portage/sys-devel/llvm-18.1.3/work/llvm/lib/Support/BLAKE3/blake3_sse2_x86-64_unix.S
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?
clang (LLVM option parsing): Unknown command line argument '-polly-parallel'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--ppc-gen-isel'?
clang (LLVM option parsing): Unknown command line argument '-polly-vectorizer=stripmine'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--slp-vectorize-hor=stripmine'?
clang (LLVM option parsing): Unknown command line argument '-polly-run-dce'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--unroll-runtime'?
clang (LLVM option parsing): Unknown command line argument '-polly-scheduling-chunksize=1'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--cache-line-size=1'?
clang (LLVM option parsing): Unknown command line argument '-polly-scheduling=dynamic'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--post-RA-scheduler=dynamic'?
clang (LLVM option parsing): Unknown command line argument '-polly-num-threads=32'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--xcore-max-threads=32'?
clang (LLVM option parsing): Unknown command line argument '-polly-omp-backend=LLVM'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--pre-RA-sched=LLVM'?
clang (LLVM option parsing): Unknown command line argument '-polly-invariant-load-hoisting'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--licm-control-flow-hoisting'?

For more info look the Gentoo PR I did for trying to package Polly for Gentoo

Edit: It is worth to say that this affects all clang versions including 19 git version.

@StormBytePP
Copy link
Author

I want to notice that the bug is still present if legacy options (-mllvm -polly ...) are used as reported initially

@Meinersbur Meinersbur mentioned this issue Oct 4, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' polly
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[driver] Do not warn about unused plugin flags. Meinersbur/llvm-project
4 participants
@StormBytePP @Meinersbur @EugeneZelenko @llvmbot