Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when I use leak-tracer, my service received segmentation fault #9

Open
iamyooon opened this issue Dec 28, 2017 · 6 comments
Open

when I use leak-tracer, my service received segmentation fault #9

iamyooon opened this issue Dec 28, 2017 · 6 comments

Comments

@iamyooon
Copy link

Hello @fredericgermain,
First of all, thanks leaktracer.
This tool is a great help to me. simple and easy!

By the way,
Sometimes our 3rd party app with leaktracer received segmentation fault and die.
below is stack trace from our internal debugging daemon.
I never seen below functions, what's that?

Backtrace :
#1  [<b6f8d604>] (request_backtrace(int) + 0x230)
#2  [<b6f8d770>] (request_backtrace(int) + 0x39c)
#3  [<b68ca7e0>] (__default_sa_restorer_v2 + 0x0)
#4  [<b69ed73c>] (_Unwind_VRS_Pop + 0x50)
#5  [<b69edfd4>] (_Unwind_Backtrace + 0x414)
#6  [<b69ed67c>] (_Unwind_VRS_Set + 0x50c)
#7  [<b69ed298>] (_Unwind_VRS_Set + 0x128)
#8  [<b69edbd8>] (_Unwind_Backtrace + 0x18)

When I run app without it, everything is ok.
Any idea?

@iamyooon
Copy link
Author

@fredericgermain
Unfortunately, I have no source code of 3rd party app.
So I can't do debugging about it now.. :(

@fredericgermain
Copy link
Owner

Each time there is an allocation, we use libgcc backtrace function to get the caller hierarchy in the runtime context.
It might not be stable depending on the environment, especially if you mix binaries compiled with different toolchains.

Can you check the Makefile and comment the part with -DUSE_BACKTRACE, maybe it works like that?

@iamyooon
Copy link
Author

iamyooon commented Dec 28, 2017

Dear @fredericgermain,
I omitted info about the environment.

arch : arm
compiler : gcc-4.8.1

I get a info from 3rd party and I make a sample program and I reproduce it.
sample program is below

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <regex.h>
#include <stdbool.h>

int main(int argc, char *argv[])
{
        regex_t reg;
        regcomp(&reg, ".", REG_EXTENDED);

        return EXIT_FAILURE;
}

And I hook segmentation fault signal by preloaded library(made by me).
signal handler call raise() to get a coredump.

gdb show me below

#0  0xb6dca584 in raise () from /home/iamyooon/rootfs/lib/libc.so.6
#1  0xb6dce01c in abort () from /home/iamyooon/rootfs/lib/libc.so.6
#2  0xb6f1b8ec in handler (sig=11) at segv.c:20
#3  <signal handler called>
#4  _Unwind_VRS_Pop (context=0xbec2f938, regclass=<optimized out>, discriminator=20464, representation=<optimized out>)
    at /home/iamyooon/gcc-4.8.1/libgcc/config/arm/unwind-arm.c:240
#5  0xb6d9afd4 in __gnu_unwind_execute (context=context@entry=0xbec2f938, uws=0xbec2f8ac, uws@entry=0xbec2f8a4) at /home/iamyooon/gcc-4.8.1/libgcc/config/arm/pr-support.c:300
#6  0xb6d9a67c in __gnu_unwind_pr_common (state=_US_VIRTUAL_UNWIND_FRAME, ucbp=0xbec2f8e0, context=0xbec2f938, id=0) at /home/iamyooon/gcc-4.8.1/libgcc/unwind-arm-common.inc:842
#7  0xb6d9a298 in __gnu_Unwind_Backtrace (trace=0xb6e818b8 <backtrace_helper>, trace_argument=0xbec2fb7c, entry_vrs=<optimized out>)
    at /home/iamyooon/gcc-4.8.1/libgcc/unwind-arm-common.inc:589
#8  0xb6d9abd8 in ___Unwind_Backtrace () at /home/iamyooon/gcc-4.8.1/libgcc/config/arm/libunwind.S:344
#9  0xb6e81a10 in backtrace () from /home/iamyooon/rootfs/lib/libc.so.6
#10 0xb6ed6310 in malloc () from /home/iamyooon/usr/lib/libleaktracer.so
#11 0xb6e54e70 in re_compile_internal () from /home/iamyooon/rootfs/lib/libc.so.6
#12 0xb6e55fac in regcomp () from /home/iamyooon/rootfs/lib/libc.so.6
#13 0x000084d8 in main ()

And I comment the -DUSE_BACKTRACE but receive segfault and die too.
I got a coredump about it.
gdb show me below.

#0  0xb6e3a584 in raise () from /home/iamyooon/lib/libc.so.6
#1  0xb6e3e01c in abort () from /home/iamyooon/lib/libc.so.6
#2  0xb6f8b8ec in handler (sig=11) at segv.c:20
#3  <signal handler called>
#4  0xb6f46300 in malloc () from /home/iamyooon/usr/lib/libleaktracer.nobt.so
#5  0xb6ec5f40 in regcomp () from /home/iamyooon/lib/libc.so.6
#6  0x000084d8 in main ()

Any idea?

@fredericgermain
Copy link
Owner

I'm not sure what happens on ARM... I tested on older MIPS and many intel toolchains, it should work though.

Can you try to define ALLOCATION_STACK_DEPTH to 1 and see if there is still a crash?

@iamyooon
Copy link
Author

iamyooon commented Jan 2, 2018

I test with below combination.

  1. use DUSE_BACKTRACE, ALLOCATION_STACK_DEPTH = 1
    -> it is ok.
  2. use DUSE_BACKTRACE, ALLOCATION_STACK_DEPTH >= 2
    -> crash
  3. not use DUSE_BACKTRACE, ALLOCATION_STACK_DEPTH = 1,2,3
    -> it is ok. but I can see only one callstack in leak.out
/tmp # cat leak.out
# LeakTracer report diff_utc_mono=315964802.529955
leak, time=514.555979, stack=0xb6f49808, size=140, data=.a...........e.......f..(f.......a..xf..xf..xf..xf
leak, time=514.555994, stack=0xb6ec7bac, size=16, data=................
leak, time=514.556010, stack=0xb6ec7bd8, size=24, data=.....................f..
  1. not use DUSE_BACKTRACE, ALLOCATION_STACK_DEPTH >= 4
    -> crash

I don't know that what's going on ARM.
Could you let me know workaround or somethhing?
thanks,

@leilu000
Copy link

Error:(288, 5) error: '_Unwind_Word' was not declared in this scope ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants