volk cross build on Xilinx e3xx fails due to incorrect processor flags

[mgarrett1955]

In building gnuradio 3.8 and volk for the ettus e3xx sdk version 4.0 (and 3.15) I get the following error:

/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/asm/neon/volk_16i_max_star_horizontal_16i.s: Assembler messages:
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/asm/neon/volk_16i_max_star_horizontal_16i.s:8: Error: selected processor does not support pld [r1,#128]' in ARM mode /home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/asm/neon/volk_16i_max_star_horizontal_16i.s:19: Error: selected processor does not support pld [r4,#128]' in ARM mode
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/asm/neon/volk_16i_max_star_horizontal_16i.s:43: Error: selected processor does not support `sxth r6,r6' in ARM mode
make[2]: *** [volk/lib/CMakeFiles/volk_obj.dir/build.make:1755: volk/lib/CMakeFiles/volk_obj.dir/__/kernels/volk/asm/neon/volk_16i_max_star_horizontal_16i.s.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:265: volk/lib/CMakeFiles/volk_obj.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

If I read the gnuradio-config_info --cflags on the e310 (uhd 4.0, gnuradio 3.8), I see the following processor switches:
-mthumb -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a9 on the e310.

I believe the build files created by Cmake should access the gnuradio/cmake/toolchains files

I am executing the following CMAKE command:

cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/gnuradio/cmake/Toolchains/oe-sdk_cross.cmake -DENABLE_GR_WXGUI=OFF -DENABLE_GR_VOCODER=OFF -DENABLE_GR_DTV=OFF -DENABLE_GR_ATSC=OFF -DENABLE_DOXYGEN=OFF -DCMAKE_INSTALL_PREFIX=/usr ../

Which point to the oe-sdk_cross.make file, in that same directory is a file called arm_cortexa8_native.cmake, Is thi supposed to be updated to an a9 processor directive?

I really appreciate any help on this matter !!

[mgarrett1955]

I am executing the following CMAKE command:

cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/gnuradio/cmake/Toolchains/oe-sdk_cross.cmake -DENABLE_GR_WXGUI=OFF -DENABLE_GR_VOCODER=OFF -DENABLE_GR_DTV=OFF -DENABLE_GR_ATSC=OFF -DENABLE_DOXYGEN=OFF -DCMAKE_INSTALL_PREFIX=/usr ../

Which points to the oe-sdk_cross.make file, in that same directory is a file called arm_cortexa8_native.cmake, Is this supposed to be updated to an a9 processor directive? Does this file need to be updated?

This is most likely a gnuradio build issue.

[balister]

Look at this commit: 92c16d1

[balister]

The oe-sdk file is the one to use with the sdk. It may need the CMAKE_ASM... directive referenced above. Generally, I use an external volk, which should also work for you.

[mgarrett1955]

Thank you for the reply!! I can see on the e310, that the arm directive it was built with is-mcpu=cortex-a9, which most likely came from the cmake/Toolchains/arm_cortex_a9_hardfp_native.cmake file during the build.

My gnuradio distribution maint-3.7, only contains the file for the a8, and that file does not contain the assembler directive!!

So I have a missing file in the main-3.7 branch, and the one that is there, does not have the assembler directive.

Thank you for the insight and for solving my problem!

[mgarrett1955]

There does not seem to be an arm_9 file in any gnuradio release, so have to add it as a cmake directive. In none of the e310 documentation I can find do they force that directive!! It is in the volk/cmake new releases, but not in the main-3.7 gnuradio/volk.

[jdemel]

I neither have the hardware nor the set up available to test any of this. Could you add all the steps to reproduce your issue please? This might help to discuss a solution and it would probably help others with the same issue if they come across this thread.
Is this an issue with GNU Radio or VOLK? Or both? Which files are you referring to?

[mgarrett1955]

Yes I will do that!! Currently my volk is in the gnuradio tree. I have not been able to figure out how to solve the issue. I added compiler switches to force the c, cpp and as flags, but recieved the same error. SOmehow Ettus created the SDK for the e31x with the correct switches but I can not figure it out. I get the same error compiling GR3.7.14 (maint-3.7) and the 4.0 Head. The files included in the new volk tree for the arm 9, do not appear in the gnuradio tree. I will update if I figure anything out. Ettus has the flags somewhere, but not in any supplied documentation for cross compiling for the e31x.

The process for building comes from this document:
https://kb.ettus.com/Software_Development_on_the_E3xx_USRP_-Building_RFNoC_UHD/GNU_Radio/_gr-ettus_from_Source

The cmake call in the document (after enabling the cross toolchain) is:

$ cd ~/rfnoc/src/gnuradio
$ mkdir build-arm
$ cd build-arm
$ cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/gnuradio/cmake/Toolchains/oe-sdk_cross.cmake -DENABLE_GR_WXGUI=OFF -DENABLE_GR_VOCODER=OFF -DENABLE_GR_DTV=OFF -DENABLE_GR_ATSC=OFF -DENABLE_DOXYGEN=OFF -DCMAKE_INSTALL_PREFIX=/usr ../
$ make -j4

Note the reference to the oe-sdk_cross.cmake file.

In the gnuradio build chain, the directory:

https://github.com/gnuradio/gnuradio/tree/master/cmake/Toolchains

Note that there is only an arm8 file (the same for maint-37).

Building the volk tip (which contains the arm9 file) might build volk correctly, but gnuradio still has to be cross compiled with the correct switches.

This is as far as I have made it...

[jdemel]

I'm still trying to understand the issue.
There's an arm9 switch that seems to be missing in a CMake toolchain file. Which one exactly? Could you add a permalink to the line with the arm9 switch in the VOLK repo?
Also, you mentioned that the arm9 switch is present in the VOLK repository but missing in the GNU Radio repository? Obviously this needs to be fixed in the correct repository.
If the problem is that GNU Radio is currently missing the a toolchain file (or it needs a fix) then this needs to go to the gnuradio repo. We can't possibly fix issues beyond our scope. Still, if there's anything missing in VOLK that would help, let's discuss this.

[mgarrett1955]

jdeml, I will try to compile volk separately. What are the cmak switches to call on the command line to force use of the arm_cortex_a9_hardfp_native.cmake compiler settings, or does it figure that out in the oe-sdk_cross.cmake script based on the cross complier configuration in the shell?

[mgarrett1955]

Just a status update:

I was able to get uhd-3.15 / gnuradio maint-3.7 volk to cross compile by adding the following flag
-DCMAKE_ASM_COMPILER_ARG1:STRING="-mcpu=cortex-a9"
(Gnuradio failed but for a different reason)

I am in the process of building a maint-3.8, UHD 4.0 cross build environment and will report back findings.

[mgarrett1955]

My SDCARD and SDK provided by Ettus, have information that I could give you is there anything I can provide you to assist on figuring this out? Let me know!

[jdemel]

Your last error message sounds like something is missing on your host. posix_memalign should probably only ever be used on MacOS at this point. Unless you use an older VOLK release.

Could you give us more details on your host? Maybe CMake is not working correctly.
Which version of VOLK do you use?
Which compiler do you use?
What where the exact steps that led to your error? Build commands are certainly useful.
Do you still try to build VOLK together with GNU Radio? In that case, could you try to build a stand-alone version of VOLK?
How did you get your copy of VOLK? Via git clone --recursive <gnuradio_repo>? In that case an update to the latest VOLK release would be helpful.

[mgarrett1955]

I will start from scratch and give you every step. Im working in a Vmware Ubutnu 18.04LTS machine currently.
Im not sure If ill start with Ubuntu 20.04.1LTS or 18.04LTS Ill let you know.

Im going to build UHD 4.0 using the Ettus SDK, Ill post the revisions.

[mgarrett1955]

A quick note, The build environment should be identical to the environment that the ETTUS sdk was built on. I'm not sure how to get the Volk version from that. I can read the gnuradio-config file. But the idea is to recreate the environment required for the SDK, to build images that are compatible with the SDK on the SD Card.

[mgarrett1955]

Attached is the full sequence starting with a fresh Ubuntu 20.04LTS.

I installed the e310 cross SDK, installed and host compiled UHD 4.0, Cross compiled UHD 4.0 and ran a test using the e310 which all succeeded.

So the UHD 4.0 cross build and SDK are all installed and working.

I then down loaded the master Volk, which succeeded for the host and failed for the cross arm build.

At the end is the results of 2 attempts the first with:

cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/volk/cmake/Toolchains/oe-sdk_cross.cmake DENABLE_DOXYGEN=OFF -DCMAKE_INSTALL_PREFIX=/usr ../

and the second with:

cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/volk/cmake/Toolchains/arm_cortex_a9_hardfp_native.cmake DENABLE_DOXYGEN=OFF -DCMAKE_INSTALL_PREFIX=/usr ../

In the second, the incorrect gcc is called for the test as indicated and annotated in the document.

Here is the installation sequence. (I misspelled Installation.. )

UHD e31x Installtion Notes.txt

[mgarrett1955]

I added this flga to the cmake: -DCMAKE_ASM_COMPILER_ARG1:STRING="-mcpu=cortex-a9", which eliminated the pld, and sxth instruction errors but left the vmov,vld2.16,,vsub,vcge,vcgt,vand,vadd and vst1 errors. These are all simd neon instructions.

I called cmake with the following directives:

cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/volk/cmake/Toolchains/oe-sdk_cross.cmake -DCMAKE_CXX_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9" -DCMAKE_C_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9" -DCMAKE_ASM_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 -g" -DENABLE_DOXYGEN=OFF -DCMAKE_INSTALL_PREFIX=/usr ../

Which actually Built!!!

I will now attempt to cross compile gnuradio maint-3.8 with a separate volk...

[mgarrett1955]

I made it through cross compiling Volk and gnuradio maint-3.8, having a separate issue with gr-ettus. I beieve there may be a python version issue against the python 3 and the python version on the e310.

Volk compiled with the above flags added to the cmake command. I will validate the operation and make sure gnuradio runs, and will post a detailed build process for the complete chain, when I solve the gr-ettus issue.

[mgarrett1955]

Here is the volk-profile run on the e310 with the sdk mounted to the device.
volk-profile-e310.txt

[mgarrett1955]

Here is the complete process all the way up to the last step which is the gr-ettus cross compile which is having an issue. This includes the Ubuntu 20.04, UHD 4.0, volk, gnuradio all host and cross compiled into the e310 sdk. The only remaining issue os the cross compile of gr-ettus. There is an issue with the cross python installation, the SDK is 3.7, gr-ettus expects 3.8 and is not finding the correct components.

VM using Ubuntu 20.04LTS-gr-ettus issue.txt

[jdemel]

VOLK is intended to run on specialized hardware and abstract that away for users. But VMs are not exactly the most reliable in passing on CPU features. We had a rather recent issue with MacOS Docker (outside Linux, Docker runs in a VM).

I assume the critical section to cross-compile VOLK is:

Cross Compile Volk:

cd ~/rfnoc/oe
source ./environment-setup-cortexa9t2hf-neon-oe-linux-gnueabi
echo $CC

arm-oe-linux-gnueabi-gcc -mthumb -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a9 --sysroot=/home/mgarrett/rfnoc/oe/sysroots/cortexa9t2hf-neon-oe-linux-gnueabi

cd ~/rfnoc/src/volk$
mkdir build-arm

----> Critical step in compiling volk for the e3xx
cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/volk/cmake/Toolchains/oe-sdk_cross.cmake -DCMAKE_CXX_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9" -DCMAKE_C_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9" -DCMAKE_ASM_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 -g" -DENABLE_DOXYGEN=OFF -DCMAKE_INSTALL_PREFIX=/usr ../

I'd like to isolate the issue here.

• Is there an issue with any of the volk/cmake/Toolchain files?
• Do you need another toolchain file?
• Your CMake flags suggest you need to overwrite most of the CMake variables that should be set. Is there something that needs to be fixed?
• What's the exact target? I know, it's an E310 but what's the exact CPU?

@balister could you chime in?

[mgarrett1955]

Here is data from the 7000 Zync manual:

The application processing unit (APU), located within the PS, contains one processor for single-core
devices or two processors for dual-core devices.

These are ARM® Cortex™-A9 processors with NEON
co-processors connected in an MP configuration sharing a 512 KB L2 cache. Each processor is a
high-performance and low-power core that implements two separate 32 KB L1 caches for instruction
and data.

The Cortex-A9 processor implements the ARM v7-A architecture with full virtual memory
support and can execute 32-bit ARM instructions, 16-bit and 32-bit Thumb instructions, and 8-bit
Java™ byte codes in the Jazelle state.

The NEON™ coprocessor media and signal processing
architecture adds instructions that target audio, video, image and speech processing, and 3D
graphics.

These advanced single instruction multiple data (SIMD) instructions are available in both
ARM and Thumb states.

[mgarrett1955]

I do not believe the Assembler switches are getting set in the current cmake structure.

I could try again with just this:
-DCMAKE_ASM_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 -g"

[mgarrett1955]

That did not work. I just can not get volk to cross compile for the Xilinx Zynq in the e310.

I tried this:
cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=~/rfnoc/src/gnuradio/cmake/Toolchains/oe-sdk_cross.cmake -DENABLE_GR_WXGUI=OFF -DENABLE_GR_VOCODER=OFF -DENABLE_GR_DTV=OFF -DENABLE_GR_ATSC=OFF -DENABLE_DOXYGEN=OFF DCMAKE_ASM_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 -g" -DCMAKE_INSTALL_PREFIX=/usr ../

WHich sets just the asm compiler switches and got this error:

[ 4%] Building C object volk/lib/CMakeFiles/volk_obj.dir/volk_machine_neonv8.c.o
In file included from /home/mgarrett/rfnoc/src/gnuradio/build-arm/volk/lib/volk_machine_neonv8.c:143:
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h: In function ‘volk_32u_byteswap_neonv8’:
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h:215:13: warning: implicit declaration of function ‘vqtbl1q_u8’; did you mean ‘vtbl1_u8’? [-Wimplicit-function-declaration]
215 | input = vqtbl1q_u8(input, idx);
| ^~~~~~~~~~
| vtbl1_u8
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h:215:13: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h:220:13: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
220 | input = vqtbl1q_u8(input, idx);

[mgarrett1955]

I am trying this again in the cmake:

-DCMAKE_CXX_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9" -DCMAKE_C_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9" -DCMAKE_ASM_FLAGS:STRING="-march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 -g"

[mgarrett1955]

Same error.

[ 4%] Building C object volk/lib/CMakeFiles/volk_obj.dir/volk_machine_neonv8.c.o
In file included from /home/mgarrett/rfnoc/src/gnuradio/build-arm/volk/lib/volk_machine_neonv8.c:143:
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h: In function ‘volk_32u_byteswap_neonv8’:
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h:215:13: warning: implicit declaration of function ‘vqtbl1q_u8’; did you mean ‘vtbl1_u8’? [-Wimplicit-function-declaration]
215 | input = vqtbl1q_u8(input, idx);
| ^~~~~~~~~~
| vtbl1_u8
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h:215:13: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
/home/mgarrett/rfnoc/src/gnuradio/volk/kernels/volk/volk_32u_byteswap.h:220:13: error: incompatible types when assigning to type ‘uint8x16_t’ from type ‘int’
220 | input = vqtbl1q_u8(input, idx);

[mgarrett1955]

THere is a cross-compiled version in the oe/sysroots sdk, and a version on the sd card.... can I probe those for the correct build calls?
How would I cross compile gnuradio with Volk turned off?

[bkerler]

I was able to sort out the issue by patching the volk/lib/CMakeLists.txt as the neonv8 detection returns invalid results and thus neonv7 needs to be enforced on the zynq. Just copy the patch file to the package/gnuradio directory in buildroot. Then gnuradio just compiles fine.
0005-force_neonv7_zynq.patch.txt