Segmentation fault when stopping playback With SigDigger develop branch

[srs4511351]

Computer:
Raspberry Pi 4
Debian GNU/Linux 12 (bookworm)
SDRplay RSP1A

develop branch
It also involves another module
https://github.com/BatchDrake/suscan/tree/develop
SigDigger 0.3.0 custom build May 23 2023 (12.2.0)
Using suscan version 0.3.0 (custom build on May 23 2023 at 01:25:58 (12.2.0))
Using sigutils version 0.3.0 (custom build on May 23 2023 at 20:42:30 (12.2.0))

When I press Run, it works normally.
When I press the Run button to stop reception, SigDigger closes and Segmentation fault appears on the terminal.

The developer of SigDigger marked my issue as third party and asked me to open an issue here.
See: BatchDrake/SigDigger#216

Here is output from gdb

Thread 1 "SigDigger" received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=) at ./malloc/malloc.c:3362
3362	./malloc/malloc.c: No such file or directory.
(gdb) bt
#0  __GI___libc_free (mem=) at ./malloc/malloc.c:3362
#1  0x0000fffff7cabd50 in SoapySDRStrings_clear ()
    at /usr/local/lib/libSoapySDR.so.0.8-2
#2  0x0000fffff7cac294 in SoapySDRArgInfo_clear ()
    at /usr/local/lib/libSoapySDR.so.0.8-2
#3  0x0000fffff7e87c78 in suscan_source_destroy (source=0xaaaaac6d8a60)
    at /home/pi/suscan/analyzer/source.c:1635
#4  0x0000fffff7e6a6b4 in suscan_local_analyzer_dtor (ptr=0xaaaaac6d85b0)
    at /home/pi/suscan/analyzer/impl/local.c:904
#5  0x0000fffff7e61e98 in suscan_analyzer_destroy (self=0xaaaaac27b9f0)
    at /home/pi/suscan/analyzer/analyzer.c:671
#6  0x0000aaaaaac51ff8 in Suscan::Analyzer::~Analyzer() ()
#7  0x0000aaaaaac52028 in Suscan::Analyzer::~Analyzer() ()
#8  0x0000aaaaaab58608 in SigDigger::Application::orderedHalt() ()
#9  0x0000aaaaaab5b084 in SigDigger::Application::onAnalyzerHalted() ()
#10 0x0000fffff681a608 in  () at /lib/aarch64-linux-gnu/libQt5Core.so.5
#11 0x0000fffff680e6cc in QObject::event(QEvent*) ()
    at /lib/aarch64-linux-gnu/libQt5Core.so.5
#12 0x0000fffff74ac0a0 in QApplicationPrivate::notify_helper(QObject*, QEvent*)
    () at /lib/aarch64-linux-gnu/libQt5Widgets.so.5
#13 0x0000fffff67dcd40 in QCoreApplication::notifyInternal2(QObject*, QEvent*)
    () at /lib/aarch64-linux-gnu/libQt5Core.so.5
#14 0x0000fffff67e00a8 in QCoreApplicationPrivate::sendPostedEvents(QObject*, in--Type  for more, q to quit, c to continue without paging--c
t, QThreadData*) () at /lib/aarch64-linux-gnu/libQt5Core.so.5
#15 0x0000fffff683f4e8 in  () at /lib/aarch64-linux-gnu/libQt5Core.so.5
#16 0x0000fffff4f5774c in g_main_context_dispatch ()
    at /lib/aarch64-linux-gnu/libglib-2.0.so.0
#17 0x0000fffff4f579e0 in  () at /lib/aarch64-linux-gnu/libglib-2.0.so.0
#18 0x0000fffff4f57a84 in g_main_context_iteration ()
    at /lib/aarch64-linux-gnu/libglib-2.0.so.0
#19 0x0000fffff683eaa8 in QEventDispatcherGlib::processEvents(QFlags) () at /lib/aarch64-linux-gnu/libQt5Core.so.5
#20 0x0000fffff67db258 in QEventLoop::exec(QFlags) () at /lib/aarch64-linux-gnu/libQt5Core.so.5
#21 0x0000fffff67e42dc in QCoreApplication::exec() ()
    at /lib/aarch64-linux-gnu/libQt5Core.so.5
#22 0x0000aaaaaab534f0 in main ()
(gdb) 

[fventuri]

@srs4511351 - thanks for reporting this problem and including the output from gdb.

Looking at that output from gdb it seems to me that this issue might be similar to what was reported here: pothosware/SoapySDR#361 - the comments there by @guruofquality and @zuckschwerdt might be helpful in understanding this problem.

Franco

[srs4511351]

@fventuri
So, is the problem in the pothosware base soapysdr release?

My crash only occurs with my SDRplay device and not with the HackRF or RTL-SDR, which may be why I was directed to SoapySDRPlay3.

After looking at pothosware/SoapySDR#361 , do you think there is a usage error by the developer of the SigDigger application?
I may not be fully understanding this issue. Should this problem be addressed by the SigDigger developer?

----Steve

[fventuri]

@srs4511351 - as you can see in the stack trace you sent, the problem appears to be originated in the method SoapySDRStrings_clear(), which is part of the SoapySDR API.

This morning I also read the latest comments on the SigDigger issue you created and I think @BatchDrake was able to fix it in the develop branch.
After you rebuild it and run it again, please let me know if you still have problems.

Franco

[srs4511351]

After the rebuild, it still fails with a seg fault when playback is stopped.

He said Now it is done exactly as how @guruofquality describes

If you look at the , There is more discussion and debug data.
I'm still not sure what software is at fault.
This problem only occurs on the SDRplay device and not with the HackRF or the RTL-SDR.
I do not have this problem with my other applications that use Soapy.

I think he is blaming the SDRplay library.

[fventuri]

@srs4511351 - if you think this is a error with the SDRplay API, I would first try to reproduce it with a simple C program (possibly based on the example at the end of SDRplay API specification guide - https://www.sdrplay.com/docs/SDRplay_API_Specification_v3.07.pdf) to rule out any other possible cause, and then open a technical support case with them (https://www.sdrplay.com/help/) so they can look into this issue in more detail.

Franco

[srs4511351]

The problem exists on all of my installed systems. My other applications work. I can't do much about it.
If it is the SDRplay API, it's the same on all of my systems. If it is defective, it's the release.

The SigDigger developer seems to say that it is the SDRplay API. His current full release version works OK, but the develop branch does not.

I would hope that you could help him. I have a lot of your software installed that is specific to SDRplay, so you must know how to handle it.
I am just trying to coordinate a fix for the SigDigger develop branch.

What can be done?

----Steve

[zuckschwerdt]

@srs4511351 a minimal test would be roughly the contents of Registration.cpp, e.g. (edit: corrected wrong API)

#include <sdrplay_api.h>
int main()
{
    sdrplay_api_ErrT err;
    // Open API
    err = sdrplay_api_Open();
    if (err != sdrplay_api_Success) {
        throw std::runtime_error("sdrplay_api_Open() failed");
    }

    unsigned int nDevs = 0;
    sdrplay_api_LockDeviceApi();
    sdrplay_api_DeviceT rspDevs[SDRPLAY_MAX_DEVICES];
    sdrplay_api_GetDevices(&rspDevs[0], &nDevs, SDRPLAY_MAX_DEVICES); // <-- fault should be here
    sdrplay_api_UnlockDeviceApi();

    // Close API
    err = sdrplay_api_Close();
    if (err != sdrplay_api_Success) {
        throw std::runtime_error("sdrplay_api_Close() failed");
    }
}

You need to run with valgrind or watch with ASAN to see sdrplay_api_GetDevices() fail.

[srs4511351]

I don't know how to do this. I haven't done any programming on this system.

If you could help me, I am interested in learning how.
The API has been released years ago. It seems like someone would have looked into this by now.

----Steve

[zuckschwerdt]

Save that content to a file playtest.c then run
g++ -ggdb -fsanitize=undefined -fsanitize=address -fno-omit-frame-pointer -Wall -I/opt/local/include -I/usr/local/include -L/opt/local/lib -L/usr/local/lib -lsdrplay_api -o playtest playtest.cpp now you can run with ./playtest

Or to use valgrind compile with just g++ -ggdb -Wall -I/opt/local/include -I/usr/local/include -L/opt/local/lib -L/usr/local/lib -lsdrplay_api -o playtest playtest.cpp and run with valgrind --track-origins=yes playtest 2> errors.log

If you don't get a crash then some other factor is influencing wether sdrplay_api_GetDevices() breaks.

[fventuri]

@zuckschwerdt - thanks for your little test program; I made a couple of changes to just use plain C (see attached); I ran it with valgrind, and this is the output:

$ valgrind --track-origins=yes ./apitest 
==23919== Memcheck, a memory error detector
==23919== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==23919== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==23919== Command: ./apitest
==23919== 
==23919== 
==23919== HEAP SUMMARY:
==23919==     in use at exit: 0 bytes in 0 blocks
==23919==   total heap usage: 19 allocs, 19 frees, 82,774 bytes allocated
==23919== 
==23919== All heap blocks were freed -- no leaks are possible
==23919== 
==23919== For lists of detected and suppressed errors, rerun with: -s
==23919== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I then used the code from SoapySDR "Basic C API example" (https://github.com/pothosware/SoapySDR/wiki/C_API_Example#basic-c-api-example), removed most of it, just left the initial device discovery part with the call to SoapySDRDeviceEnumerate() (code attached), and ran it with valgrind:

$ valgrind --track-origins=yes ./soapytest 
==23986== Memcheck, a memory error detector
==23986== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==23986== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==23986== Command: ./soapytest
==23986== 
Found device #0: driver=sdrplay, label=SDRplay Dev0 RSPdx **********, serial=**********, 
Done
==23986== 
==23986== HEAP SUMMARY:
==23986==     in use at exit: 6,600 bytes in 16 blocks
==23986==   total heap usage: 119 allocs, 103 frees, 131,147 bytes allocated
==23986== 
==23986== LEAK SUMMARY:
==23986==    definitely lost: 0 bytes in 0 blocks
==23986==    indirectly lost: 0 bytes in 0 blocks
==23986==      possibly lost: 0 bytes in 0 blocks
==23986==    still reachable: 6,600 bytes in 16 blocks
==23986==         suppressed: 0 bytes in 0 blocks
==23986== Rerun with --leak-check=full to see details of leaked memory
==23986== 
==23986== For lists of detected and suppressed errors, rerun with: -s
==23986== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I ran it again with the additional options --leak-check=full --show-leak-kinds=all, and this is the output:

$ valgrind --track-origins=yes --leak-check=full --show-leak-kinds=all ./soapytest 
==23872== Memcheck, a memory error detector
==23872== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==23872== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==23872== Command: ./soapytest
==23872== 
Found device #0: driver=sdrplay, label=SDRplay Dev0 RSPdx **********, serial=**********, 
Done
==23872== 
==23872== HEAP SUMMARY:
==23872==     in use at exit: 6,600 bytes in 16 blocks
==23872==   total heap usage: 119 allocs, 103 frees, 131,147 bytes allocated
==23872== 
==23872== 60 bytes in 1 blocks are still reachable in loss record 1 of 7
==23872==    at 0x484182F: malloc (vg_replace_malloc.c:431)
==23872==    by 0x402468F: malloc (rtld-malloc.h:56)
==23872==    by 0x402468F: strdup (strdup.c:42)
==23872==    by 0x40089FB: _dl_map_object (dl-load.c:2186)
==23872==    by 0x400C62B: dl_open_worker_begin (dl-open.c:534)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400BDDF: dl_open_worker (dl-open.c:782)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400C1B3: _dl_open (dl-open.c:884)
==23872==    by 0x4F8A6D3: dlopen_doit (dlopen.c:56)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x4001678: _dl_catch_error (dl-catch.c:256)
==23872==    by 0x4F8A1B2: _dlerror_run (dlerror.c:138)
==23872== 
==23872== 60 bytes in 1 blocks are still reachable in loss record 2 of 7
==23872==    at 0x484182F: malloc (vg_replace_malloc.c:431)
==23872==    by 0x400BB50: UnknownInlinedFun (rtld-malloc.h:56)
==23872==    by 0x400BB50: _dl_new_object (dl-object.c:199)
==23872==    by 0x4006FEE: _dl_map_object_from_fd (dl-load.c:1063)
==23872==    by 0x4008AA8: _dl_map_object (dl-load.c:2253)
==23872==    by 0x400C62B: dl_open_worker_begin (dl-open.c:534)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400BDDF: dl_open_worker (dl-open.c:782)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400C1B3: _dl_open (dl-open.c:884)
==23872==    by 0x4F8A6D3: dlopen_doit (dlopen.c:56)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x4001678: _dl_catch_error (dl-catch.c:256)
==23872== 
==23872== 81 bytes in 3 blocks are still reachable in loss record 3 of 7
==23872==    at 0x484182F: malloc (vg_replace_malloc.c:431)
==23872==    by 0x402468F: malloc (rtld-malloc.h:56)
==23872==    by 0x402468F: strdup (strdup.c:42)
==23872==    by 0x40144A7: _dl_load_cache_lookup (dl-cache.c:515)
==23872==    by 0x4008D13: _dl_map_object (dl-load.c:2120)
==23872==    by 0x400282C: openaux (dl-deps.c:64)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x4002C87: _dl_map_object_deps (dl-deps.c:232)
==23872==    by 0x400C694: dl_open_worker_begin (dl-open.c:592)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400BDDF: dl_open_worker (dl-open.c:782)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400C1B3: _dl_open (dl-open.c:884)
==23872== 
==23872== 81 bytes in 3 blocks are still reachable in loss record 4 of 7
==23872==    at 0x484182F: malloc (vg_replace_malloc.c:431)
==23872==    by 0x400BB50: UnknownInlinedFun (rtld-malloc.h:56)
==23872==    by 0x400BB50: _dl_new_object (dl-object.c:199)
==23872==    by 0x4006FEE: _dl_map_object_from_fd (dl-load.c:1063)
==23872==    by 0x4008AA8: _dl_map_object (dl-load.c:2253)
==23872==    by 0x400282C: openaux (dl-deps.c:64)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x4002C87: _dl_map_object_deps (dl-deps.c:232)
==23872==    by 0x400C694: dl_open_worker_begin (dl-open.c:592)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400BDDF: dl_open_worker (dl-open.c:782)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400C1B3: _dl_open (dl-open.c:884)
==23872== 
==23872== 1,276 bytes in 1 blocks are still reachable in loss record 5 of 7
==23872==    at 0x484682C: calloc (vg_replace_malloc.c:1554)
==23872==    by 0x400B86D: UnknownInlinedFun (rtld-malloc.h:44)
==23872==    by 0x400B86D: _dl_new_object (dl-object.c:92)
==23872==    by 0x4006FEE: _dl_map_object_from_fd (dl-load.c:1063)
==23872==    by 0x4008AA8: _dl_map_object (dl-load.c:2253)
==23872==    by 0x400C62B: dl_open_worker_begin (dl-open.c:534)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400BDDF: dl_open_worker (dl-open.c:782)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400C1B3: _dl_open (dl-open.c:884)
==23872==    by 0x4F8A6D3: dlopen_doit (dlopen.c:56)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x4001678: _dl_catch_error (dl-catch.c:256)
==23872== 
==23872== 1,344 bytes in 4 blocks are still reachable in loss record 6 of 7
==23872==    at 0x484682C: calloc (vg_replace_malloc.c:1554)
==23872==    by 0x401375F: UnknownInlinedFun (rtld-malloc.h:44)
==23872==    by 0x401375F: _dl_check_map_versions (dl-version.c:280)
==23872==    by 0x400C6DA: dl_open_worker_begin (dl-open.c:600)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400BDDF: dl_open_worker (dl-open.c:782)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400C1B3: _dl_open (dl-open.c:884)
==23872==    by 0x4F8A6D3: dlopen_doit (dlopen.c:56)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x4001678: _dl_catch_error (dl-catch.c:256)
==23872==    by 0x4F8A1B2: _dlerror_run (dlerror.c:138)
==23872==    by 0x4F8A78E: UnknownInlinedFun (dlopen.c:71)
==23872==    by 0x4F8A78E: dlopen@@GLIBC_2.34 (dlopen.c:81)
==23872== 
==23872== 3,698 bytes in 3 blocks are still reachable in loss record 7 of 7
==23872==    at 0x484682C: calloc (vg_replace_malloc.c:1554)
==23872==    by 0x400B86D: UnknownInlinedFun (rtld-malloc.h:44)
==23872==    by 0x400B86D: _dl_new_object (dl-object.c:92)
==23872==    by 0x4006FEE: _dl_map_object_from_fd (dl-load.c:1063)
==23872==    by 0x4008AA8: _dl_map_object (dl-load.c:2253)
==23872==    by 0x400282C: openaux (dl-deps.c:64)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x4002C87: _dl_map_object_deps (dl-deps.c:232)
==23872==    by 0x400C694: dl_open_worker_begin (dl-open.c:592)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400BDDF: dl_open_worker (dl-open.c:782)
==23872==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==23872==    by 0x400C1B3: _dl_open (dl-open.c:884)
==23872== 
==23872== LEAK SUMMARY:
==23872==    definitely lost: 0 bytes in 0 blocks
==23872==    indirectly lost: 0 bytes in 0 blocks
==23872==      possibly lost: 0 bytes in 0 blocks
==23872==    still reachable: 6,600 bytes in 16 blocks
==23872==         suppressed: 0 bytes in 0 blocks
==23872== 
==23872== For lists of detected and suppressed errors, rerun with: -s
==23872== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

My tests were run on my x86_64 Linux desktop running Fedora Core and SDRplay API version 3.07.

@srs4511351 - Steve, I saw in your initial message that you are running the application that crashes on a Raspberry Pi with 64bit ARM (aarch64), so your shared library for the SDRplay API is different from mine. Since they are different, it is also possible that there's a bug in your version of that shared library that is not in the one for x86_64 that I am running here.

Finally since the SDRplay API library is proprietary of SDRplay and it is not open source, there's only so much that I/we can do if the issue turns out to be in one of the functions of the API itself, like sdrplay_api_GetDevices().
In that case I think the only way to get it resolved is to open a technical support case with SDRplay.

Franco

apitest.zip
soapytest.zip

[srs4511351]

@zuckschwerdt
I would like to try this on my Raspbery Pi, but I was unable to compile your code. As instructed, I saved as playtest.c
The compile command referred to playtest.cpp

Running the compile with the file named playtest.cpp:

playtest.cpp: In function ‘int main()’:
playtest.cpp:8:20: error: ‘runtime_error’ is not a member of ‘std’
    8 |         throw std::runtime_error("sdrplay_api_Open() failed");
      |                    ^~~~~~~~~~~~~
playtest.cpp:20:20: error: ‘runtime_error’ is not a member of ‘std’
   20 |         throw std::runtime_error("sdrplay_api_Close() failed");
      |                    ^~~~~~~~~~~~~

@fventuri
Not really knowing how to compile your code, I tried to compile apitest.c with the @zuckschwerdt command:

g++ -ggdb -fsanitize=undefined -fsanitize=address -fno-omit-frame-pointer -Wall -I/opt/local/include -I/usr/local/include -L/opt/local/lib -L/usr/local/lib -lsdrplay_api -o apitest apitest.c

I got

/usr/bin/ld: /tmp/ccBSyQhH.o: in function `main':
/home/pi/apitest/apitest.c:9: undefined reference to `sdrplay_api_Open'
/usr/bin/ld: /home/pi/apitest/apitest.c:16: undefined reference to `sdrplay_api_LockDeviceApi'
/usr/bin/ld: /home/pi/apitest/apitest.c:18: undefined reference to `sdrplay_api_GetDevices'
/usr/bin/ld: /home/pi/apitest/apitest.c:22: undefined reference to `sdrplay_api_UnlockDeviceApi'
/usr/bin/ld: /home/pi/apitest/apitest.c:25: undefined reference to `sdrplay_api_Close'
collect2: error: ld returned 1 exit status

----Steve

[zuckschwerdt]

@fventuri I wasn't sure if the C++ runtime is contributing to this strangeness. I don't see any fault. The code is just a guess based on what is seen in BatchDrake/SigDigger#216 (comment)
i.e.

==2340== Source and destination overlap in memcpy(0xc49f570, 0x487e010, 88887341568)
==2340==    by 0x1E517C6F: memcpy (string3.h:53)
==2340==    by 0x1E517C6F: sdrplay_api_GetDevices (sdrplay_api.cpp:321)

I would suspect an unterminated string that only gets out of bounds because valgrind filled the memory with guard values or such heisenbug effects?

[zuckschwerdt]

@srs4511351 sorry, add a first line of #include <stdexcept>