Commonly observed issues with the build environment

[kumaakh]

@mraduldubey @Vinayak-YB @mohammedzakikochargi lets update this discussion with commonly observed issues that we see after checkout and build. I'll start:

1. Windows. Issue: cmake does not find CUDA!

Possible causes/solutions:

1. Use the correct (latest) cmake version
2. Make sure CUDA toolkit is installed after installing visual studio. if for some reason you had to install CUDA toolkit before VS, please do the following:
   Copy the four files from this extracted directory .\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\extras\visual_studio_integration\MSBuildExtensions into the MSBuild folder of your VS install C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\BuildCustomizations
   The four files are:

• CUDA 10.2 props,
• CUDA 10.2 targets,
• CUDA 10.2 xml, and
• Nvda.Build.CudaTasks.v10.2.dll

[kumaakh]

2. Ubuntu 18.04/20.04 on WSL with CUDA 10.2./11.7 does not find CUDA

The following has been tested with Ubuntu 20.04 on wsl and CUDA 11.7 but it should work as is for Ubuntu 18.04 and CUDA 10.2 as well

• Follow instructions to enable CUDA on WSL from here
• Add CUDA in path in ~/.profile export PATH=/usr/local/cuda-11.7/bin:$PATH
• test by running nvcc --version
• install cudnn by
  • downloading cudnn 8.5 from here. Note you need to login to Nvidia website for this. So download to windows and then cp /mnt/c/Users/<userName>/Downloads/cudnn-linux-x86_64-8.5.0.96_cuda11-archive.tar.xz .
  • Now follow the instructions from here

In my installation I found that cmake 3.24 was failing to determine the CMAKE_CUDA_ARCHITECTURE, hence I forced it by adding the following line to CMakeFilesList.txt: set (CMAKE_CUDA_ARCHITECTURES 52 60 70 75)

[kumaakh]

3. I could build and run with NO_CUDA flavour but now when I try to build with CUDA I get openCV errors : "cuda_filters.hpp not found"

• This is an issue with vcpkg: when you built NO_CUDA flavour, we used opencv features ["nonfree","png","jpeg","tiff","dnn","webp","contrib"] from vcpkg.nocuda.json file and vcpkg cached that build
• Now when you build with cuda using vcpkg.cuda.json, even though the opencv features are different, vcpkg assumes that the cached build will suffice and does not rebuild it.
• So you can delete ypur cache from ~/.cache/vcpkg/archives or %LOCALAPPDATA%\vcpkg\archives and incur a full rebuild for next few hours or

- You can surgically remove opencv from archieves:

• find the larget archieve in terms of size : du -a ~/.cache/vcpkg/archives | sort -n -r | head -n 10 that is typically the opencv archieve
• confirm by running unzip -l ~/.cache/vcpkg/archives/xx/xxxxxxxxxxxxx.zip for the correct file: you should see some openCV files and folders
• finally just remove that file rm ~/.cache/vcpkg/archives/xx/xxxxxxxxxxxxx.zip
• Next build will not find opencv in vcpkg archieve and will rebuild it with CUDA

[kumaakh]

4. Windows build fails with the following error:

CMake Error at C:/akhil/git/ApraPipes/_build/vcpkg_installed/x64-windows/share/boost-build/boost-modular-build.cmake:336 (message):
Unsupported value for VCPKG_PLATFORM_TOOLSET: 'v143'

As of Sept 2022. We are using VS 2019 for building Apra Pipes. The above error shows that you are using VS 2022. So please install VS2019. Unfortunately vcpkg as-is picks the latest VS installation so if you have both 2019 and 2022, please uninstall VS2022 till we find a way to eclipse VS2022 from vcpkg scripts.

At some near future we will upgrade the whole stack to VS2022, Cuda 11.x and related tools.

[kumaakh]

5. Windows CUDA builder cribs about not finding CUDA with an error like the following

Building package cuda[core]:x64-windows...
-- Installing port from location: C:\a\w\ApraPipes\ApraPipes\vcpkg\buildtrees\versioning_\versions\cuda\43b8008115ab934b4cf1b36c38bb02ed845712bd
-- Executing NVCC-NOTFOUND --version resulted in error: 1
CMake Error at buildtrees/versioning_/versions/cuda/43b8008115ab934b4cf1b36c38bb02ed845712bd/vcpkg_find_cuda.cmake:67 (message):
Could not find CUDA. Before continuing, please download and install CUDA
(v10.1.0 or higher) from:

This can be solved by making sure that environment variable "CUDA_PATH" points to the folder where nvcc.exe lives: e.g
set CUDA_BIN_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin
set CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\

On a build agent the same is acheived by the following like in workflow yam file .github/workflows/build-test-win.yml:
echo "CUDA_PATH=c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin" >> $GITHUB_ENV

[mohammedzakikochargi]

If the above solution does not work, try the below .
In vcpkg/ports/cuda/vcpkg_find_cuda.cmake, At the top of the file hardcode the cuda path as given below depending upon cuda version used.
set(ENV{CUDA_BIN_PATH} "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2/bin")

[Vinayak-YB]

6. Video_Codec_SDK Error:
   While building aprapipes if there is an issue which points to libnvcuvid.so file in Video Codec SDK. This may have occurred due to error by git lfs and can be solved by cloning it again. To verify check md5sum of libnvcuvid.so in path
   /ApraPipes/thirdparty/Video_Codec_SDK_10.0.26/Lib/linux/stubs/x86_64/ - for x86 systems. It must match with the value given below then try to build aprapipes again.
   bcfd87f2c83cfa834e60b8e4aeb45479 libnvcuvid.so

[mohammedzakikochargi]

7. OpenH264 build issue on arm64 - (Ninja)

1. Remove the ninja-build from the system - sudo apt-get remove ninja-build.
2. Download ninja using https://github.com/ninja-build/ninja/archive/refs/tags/v1.10.2.tar.gz
3. Extract and run ./configure.py
4. RUN - mkdir build ; cd build ; cmake .. ; make
5. sudo mv ninja /usr/local/bin ; sudo chmod +x /usr/local/bin/ninja
6. Delete all generated dir in vcpkg folder i.e. buildtrees, downloads and packages
7. Rebuilt again