L4T 5.1 missing return statement in encoder causing segmentation fault

[bmegli]

L4T version

After flashing L4T 35.1 with SDK Manger

cat /etc/nv_tegra_release 
# R35 (release), REVISION: 2.1, GCID: 32413640, BOARD: t186ref, EABI: aarch64, DATE: Tue Jan 24 23:38:33 UTC 2023

apt-cache show nvidia-l4t-core

Package: nvidia-l4t-core
Version: 35.2.1-20230124153320

hardware

AGX Orin 32 GB

jetson-ffmpeg version

From fork:

• Migrating from nvbuf_utils to NvUtils. #125
• notably encoder's code is not changed there

FFmpeg version

n4.2.7 (git tag)

build

Building jetson-ffmpeg for release

# ...
cmake -DCMAKE_BUILD_TYPE=Release ..
make
# ...

symptoms

On closing encoder:

• a series of messages
  • Error while dequeing buffer from output plane
• followed by segmentation fualt

previous behavior

FWIW this does not happen for me in L4T 34.1.

But that's just C and C++ UB (Undefined Behavior)

[bmegli]

Fix

Obviously add missing return statement like here

[bmegli]

Similar problem manifests in decoder's

• void *dec_capture_loop_fcn(void *arg)
• function was by mistake declared as returning void pointer
• but it returns nothing

This is UB. In release build and with versions as stated above it also leads to segmentation fault (this time during decoding).

The fix is to declare function as returning void correctly

[eusoubrasileiro]

@bmegli thank you for pointing another repo more updated.
Found also this https://github.com/Keylost/jetson-ffmpeg
Would be nice I there could be an updated version here instead of multiple branches. Thank you so much regardless.