Errors building for riscv32 architecture (binutils bug?)

[sellicott]

Hi all,
I just tried building pk for the rv32im architecture and encountered the following issues. Some of these are duplicates of other issues, but I wanted to consolidate them for better searchability, and for the sake of completeness.

My host computer is set up as follows
OS: Arch Linux
gcc: riscv64-elf-gcc (Arch Linux Repositories) 12.2.0
ld: GNU ld (GNU Binutils) 2.39

Error 1: Needing to manually add the _zicsr and _zifencei options to the architecture flag.

Following these build steps

mkdir build32-clean
cd build32-clean
../configure \
--prefix=$HOME/.local \
--host=riscv64-elf \
--with-arch=rv32im
make

This produces the following errors:

riscv64-elf-gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks -fno-PIE  -march=rv32im -mabi=ilp32 -DBBL_LOGO_FILE=\"bbl_logo_file\" -DMEM_START=0x80000000 -fno-stack-protector -U_FORTIFY_SOURCE -DBBL_PAYLOAD=\"bbl_payload\" -I. -I../pk -I../bbl -I../softfloat -I../dummy_payload -I../machine -I../util -c ../pk/syscall.c
../pk/pk.h: Assembler messages:
../pk/pk.h:59: Error: unrecognized opcode `csrr a5,timeh', extension `zicsr' required
../pk/pk.h:60: Error: unrecognized opcode `csrr a3,time', extension `zicsr' required
../pk/pk.h:61: Error: unrecognized opcode `csrr a4,timeh', extension `zicsr' required
../pk/pk.h:75: Error: unrecognized opcode `csrr a5,cycleh', extension `zicsr' required
../pk/pk.h:76: Error: unrecognized opcode `csrr a1,cycle', extension `zicsr' required
../pk/pk.h:77: Error: unrecognized opcode `csrr a4,cycleh', extension `zicsr' required
../pk/pk.h:91: Error: unrecognized opcode `csrr a5,instreth', extension `zicsr' required
../pk/pk.h:92: Error: unrecognized opcode `csrr a1,instret', extension `zicsr' required
../pk/pk.h:93: Error: unrecognized opcode `csrr a4,instreth', extension `zicsr' required
../pk/pk.h:75: Error: unrecognized opcode `csrr a1,cycleh', extension `zicsr' required
../pk/pk.h:76: Error: unrecognized opcode `csrr a0,cycle', extension `zicsr' required
../pk/pk.h:77: Error: unrecognized opcode `csrr a5,cycleh', extension `zicsr' required
../pk/pk.h:75: Error: unrecognized opcode `csrr a1,cycleh', extension `zicsr' required
../pk/pk.h:76: Error: unrecognized opcode `csrr a0,cycle', extension `zicsr' required
../pk/pk.h:77: Error: unrecognized opcode `csrr a5,cycleh', extension `zicsr' required
../pk/pk.h:75: Error: unrecognized opcode `csrr s0,cycleh', extension `zicsr' required
../pk/pk.h:76: Error: unrecognized opcode `csrr s1,cycle', extension `zicsr' required
../pk/pk.h:77: Error: unrecognized opcode `csrr a5,cycleh', extension `zicsr' required
../pk/pk.h:75: Error: unrecognized opcode `csrr s0,cycleh', extension `zicsr' required
../pk/pk.h:76: Error: unrecognized opcode `csrr s1,cycle', extension `zicsr' required
../pk/pk.h:77: Error: unrecognized opcode `csrr a5,cycleh', extension `zicsr' required
make: *** [Makefile:336: syscall.o] Error 1

I realize this has been mentioned before (#260, #285, spike #1140) but I think this should be documented in the readme.

The settings were updated as follows:

../configure \
--prefix=$HOME/.local \
--host=riscv64-elf \
--with-arch=rv32im_zifencei_zicsr
make

Error 2: Problems linking the compiled object files

Reruning configure and make with the updated architecture settings produces a clean compile, but the following error for linking:

riscv64-elf-gcc -Wl,--build-id=none -nostartfiles -nostdlib -static  -march=rv32im_zifencei_zicsr -mabi=ilp32 -fno-stack-protector -o pk pk.o -L.  -lpk  -lmachine  -lsoftfloat  -lutil -lgcc -Wl,--defsym=MEM_START=0x80000000,-T,../pk/pk.lds
/usr/lib/gcc/riscv64-elf/12.2.0/../../../../riscv64-elf/bin/ld: /usr/lib/gcc/riscv64-elf/12.2.0/libgcc.a(div.o): ABI is incompatible with that of the selected emulation:
  target emulation `elf64-littleriscv' does not match `elf32-littleriscv'
/usr/lib/gcc/riscv64-elf/12.2.0/../../../../riscv64-elf/bin/ld: failed to merge target specific data of file /usr/lib/gcc/riscv64-elf/12.2.0/libgcc.a(div.o)
collect2: error: ld returned 1 exit status
make: *** [Makefile:336: pk] Error 1

This error is sort of insidious and comes from the fact the linker doesn't recognize the _zicsr and _zifencei options and silently defaults to riscv64 when it doesn't know what to do, making it fail to build the resulting binary. reference link

My hacky workaround for this is to manually run the linking step with --march=rv32im, then running make again.

riscv64-elf-gcc -Wl,--build-id=none -nostartfiles -nostdlib -static  -march=rv32im -mabi=ilp32 -fno-stack-protector -o pk pk.o -L.  -lpk  -lmachine  -lsoftfloat  -lutil -lgcc -Wl,--defsym=MEM_START=0x80000000,-T,../pk/pk.lds
make

at this point my binaries are compiled.

I think this second issue is really a problem with binutils, not this project. However, I thought you should be aware of it.

Thanks,
-Sam Ellicott
Soli Deo Gloria

[sellicott]

I believe #275 stems from the linker issue.

[kele14x]

Got same issue, build and use a pure 32-bit gcc toolchain (riscv32-unknown-elf-*) seems work.