using dlsym instead of __libc_malloc will deadlock because of malloc happens before startMonitoringAllThreads

[ZhuChasel]

Hi,
I use dlsym(RTLD_NEXT) instead of using unambiguous function name like "__libc_malloc".
But it seems to be deadlock durring lib link.

show the gdb backtrace:
(gdb) bt
#0 pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:94
#1 0x00007f2bbf265c57 in leaktracer::MemoryTrace::Setup() () from /root/LeakTracer/LeakTracer/build/x86_64-linux-gnu/4.8/libleaktracer.so
#2 0x00007f2bbf2655e9 in calloc () from /root/LeakTracer/LeakTracer/build/x86_64-linux-gnu/4.8/libleaktracer.so
#3 0x00007f2bbe30d690 in _dlerror_run (operate=operate@entry=0x7f2bbe30d130 <dlsym_doit>, args=args@entry=0x7fff6832b590) at dlerror.c:141
#4 0x00007f2bbe30d198 in __dlsym (handle=, name=) at dlsym.c:70
#5 0x00007f2bbf2659df in leaktracer::MemoryTrace::init_no_alloc_allowed() ()
from /root/LeakTracer/LeakTracer/build/x86_64-linux-gnu/4.8/libleaktracer.so
#6 0x00007f2bbf050a80 in pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:103
#7 0x00007f2bbf265c57 in leaktracer::MemoryTrace::Setup() () from /root/LeakTracer/LeakTracer/build/x86_64-linux-gnu/4.8/libleaktracer.so
#8 0x00007f2bbf264de9 in malloc () from /root/LeakTracer/LeakTracer/build/x86_64-linux-gnu/4.8/libleaktracer.so
#9 0x00007f2bbed6d566 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007f2bbf4fa1da in call_init (l=, argc=argc@entry=1, argv=argv@entry=0x7fff6832b6b8, env=env@entry=0x7fff6832b6c8)
at dl-init.c:78
#11 0x00007f2bbf4fa2c3 in call_init (env=, argv=, argc=, l=) at dl-init.c:36
#12 _dl_init (main_map=0x7f2bbf70e1c8, argc=1, argv=0x7fff6832b6b8, env=0x7fff6832b6c8) at dl-init.c:126
#13 0x00007f2bbf4eb29a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#14 0x0000000000000001 in ?? ()
#15 0x00007fff6832c825 in ?? ()
#16 0x0000000000000000 in ?? ()

lib link as below:
Reading symbols from test...done.
Attaching to program: /root/LeakTracer/LeakTracer/tests/test, process 26911
Reading symbols from /root/LeakTracer/LeakTracer/build/x86_64-linux-gnu/4.8/libleaktracer.so...(no debugging symbols found)...done.
Loaded symbols for /root/LeakTracer/LeakTracer/build/x86_64-linux-gnu/4.8/libleaktracer.so
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libpthread-2.19.so...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Loaded symbols for /lib/x86_64-linux-gnu/libpthread.so.0
Reading symbols from /usr/lib/x86_64-linux-gnu/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libstdc++.so.6
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libgcc_s.so.1
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libc.so.6
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/librt-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/librt.so.1
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libdl-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libdl.so.2
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/ld-2.19.so...done.
done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libm-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libm.so.6
pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:94
94 ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S: No such file or directory.

It seems that malloc happens before completed lib link, and _dlerror_run when trying dlysm. But _dlerror_run need to calloc, so deadlock happens.
Any advise? I use test.cc as the demo.

[fredericgermain]

Hi,

__dlsym(RTLD_NEXT) here calls calloc, hence cannot be used in leaktracer::MemoryTrace::init_no_alloc_allowed(), otherwise you get the deadlock you're in.

I'm not sure what you want to do by calling __dlsym(RTLD_NEXT) ?