AddressSanitizer container-overflow: is this a spdlog bug or a coding mistake?

[viniciusferrao]

Hello, I've stumbled with the following issue at the end of my code:

==20472==ERROR: AddressSanitizer: container-overflow on address 0x6020000004d8 at pc 0x0001027f0c9f bp 0x7ff7bdb4ae10 sp 0x7ff7bdb4ae08

(...)

SUMMARY: AddressSanitizer: container-overflow memory:3237 in std::__1::shared_ptrspdlog::sinks::sink::~shared_ptr()

I'm trying to understand if this is:

• A legitimate bug on spdlog
• A false positive from AddressSanitizer (Apple clang version 13.0.0 (clang-1300.0.29.30)
• Coding mistake by myself

If it's a bug I'll open a proper bug report.

My code is available specifically here:
https://github.com/viniciusferrao/cloysterhpc/blob/master/src/services/log.cpp
https://github.com/viniciusferrao/cloysterhpc/blob/master/src/services/log.h
https://github.com/viniciusferrao/cloysterhpc/blob/master/src/main.cpp#L29

If someone is willing to help I'm very grateful.
Thanks.

PS: Full output of the AddressSanitizer

==20472==ERROR: AddressSanitizer: container-overflow on address 0x6020000004d8 at pc 0x0001027f0c9f bp 0x7ff7bdb4ae10 sp 0x7ff7bdb4ae08
READ of size 8 at 0x6020000004d8 thread T0
    #0 0x1027f0c9e in std::__1::shared_ptr<spdlog::sinks::sink>::~shared_ptr() memory:3237
    #1 0x1027ef2d8 in std::__1::shared_ptr<spdlog::sinks::sink>::~shared_ptr() memory:3236
    #2 0x1027f0930 in void std::__1::destroy_at<std::__1::shared_ptr<spdlog::sinks::sink> >(std::__1::shared_ptr<spdlog::sinks::sink>*) base.h:118
    #3 0x1027f08d0 in void std::__1::allocator_traits<std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> > >::destroy<std::__1::shared_ptr<spdlog::sinks::sink>, void, void>(std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> >&, std::__1::shared_ptr<spdlog::sinks::sink>*) allocator_traits.h:315
    #4 0x1027f0827 in std::__1::__vector_base<std::__1::shared_ptr<spdlog::sinks::sink>, std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> > >::__destruct_at_end(std::__1::shared_ptr<spdlog::sinks::sink>*) vector:428
    #5 0x1027f0614 in std::__1::__vector_base<std::__1::shared_ptr<spdlog::sinks::sink>, std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> > >::clear() vector:371
    #6 0x1027eff1f in std::__1::__vector_base<std::__1::shared_ptr<spdlog::sinks::sink>, std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> > >::~__vector_base() vector:465
    #7 0x1028beb98 in spdlog::logger::~logger()+0x58 (main:x86_64+0x10050ab98)
    #8 0x1027fdfee in void std::__1::destroy_at<spdlog::logger>(spdlog::logger*) base.h:118
    #9 0x1027fde90 in void std::__1::allocator_traits<std::__1::allocator<spdlog::logger> >::destroy<spdlog::logger, void, void>(std::__1::allocator<spdlog::logger>&, spdlog::logger*) allocator_traits.h:315
    #10 0x1027fa0c3 in std::__1::__shared_ptr_emplace<spdlog::logger, std::__1::allocator<spdlog::logger> >::__on_zero_shared() memory:2642
    #11 0x1028aac69 in spdlog::details::registry::drop_all()+0x51 (main:x86_64+0x1004f6c69)
    #12 0x1028aacce in spdlog::details::registry::shutdown()+0x3a (main:x86_64+0x1004f6cce)
    #13 0x1027efc20 in Log::shutdown() log.cpp:53
    #14 0x10270fc7a in main main.cpp:83
    #15 0x110ccc4fd in start+0x1cd (dyld:x86_64+0x54fd)

0x6020000004d8 is located 8 bytes inside of 16-byte region [0x6020000004d0,0x6020000004e0)
allocated by thread T0 here:
    #0 0x103d458fd in wrap__Znwm+0x7d (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x508fd)
    #1 0x1027f247c in void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) new:235
    #2 0x1027f2254 in std::__1::__libcpp_allocate(unsigned long, unsigned long) new:261
    #3 0x1027f6aab in std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> >::allocate(unsigned long) memory:870
    #4 0x1027f6104 in std::__1::allocator_traits<std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> > >::allocate(std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> >&, unsigned long) allocator_traits.h:260
    #5 0x1027f525a in std::__1::vector<std::__1::shared_ptr<spdlog::sinks::sink>, std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> > >::__vallocate(unsigned long) vector:993
    #6 0x1028b9653 in std::__1::vector<std::__1::shared_ptr<spdlog::sinks::sink>, std::__1::allocator<std::__1::shared_ptr<spdlog::sinks::sink> > >::vector<std::__1::shared_ptr<spdlog::sinks::sink> const*>(std::__1::shared_ptr<spdlog::sinks::sink> const*, std::__1::enable_if<(__is_cpp17_forward_iterator<std::__1::shared_ptr<spdlog::sinks::sink> const*>::value) && (is_constructible<std::__1::shared_ptr<spdlog::sinks::sink>, std::__1::iterator_traits<std::__1::shared_ptr<spdlog::sinks::sink> const*>::reference>::value), std::__1::shared_ptr<spdlog::sinks::sink> const*>::type)+0x31 (main:x86_64+0x100505653)
    #7 0x1028b1010 in spdlog::logger::logger<std::__1::shared_ptr<spdlog::sinks::sink> const*>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::shared_ptr<spdlog::sinks::sink> const*, std::__1::shared_ptr<spdlog::sinks::sink> const*)+0x46 (main:x86_64+0x1004fd010)
    #8 0x1028b1fff in spdlog::logger::logger(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::initializer_list<std::__1::shared_ptr<spdlog::sinks::sink> >)+0x37 (main:x86_64+0x1004fdfff)
    #9 0x1028b1f63 in spdlog::logger::logger(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::shared_ptr<spdlog::sinks::sink>)+0x5b (main:x86_64+0x1004fdf63)
    #10 0x1028aba97 in spdlog::details::registry::registry()+0x1a3 (main:x86_64+0x1004f7a97)
    #11 0x1028a9df1 in spdlog::details::registry::instance()+0x39 (main:x86_64+0x1004f5df1)
    #12 0x1028aa8a7 in spdlog::register_logger(std::__1::shared_ptr<spdlog::logger>)+0x10 (main:x86_64+0x1004f68a7)
    #13 0x1027ee95e in Log::init(Log::Level) log.cpp:49
    #14 0x10270f240 in main main.cpp:29
    #15 0x110ccc4fd in start+0x1cd (dyld:x86_64+0x54fd)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow memory:3237 in std::__1::shared_ptr<spdlog::sinks::sink>::~shared_ptr()
Shadow bytes around the buggy address:
  0x1c0400000040: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x1c0400000050: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x1c0400000060: fa fa fd fa fa fa fd fd fa fa fd fd fa fa fd fd
  0x1c0400000070: fa fa fd fd fa fa 00 00 fa fa 00 fa fa fa fd fa
  0x1c0400000080: fa fa fd fa fa fa fd fd fa fa fd fd fa fa fd fd
=>0x1c0400000090: fa fa 00 fa fa fa 00 fa fa fa fc[fc]fa fa 00 00
  0x1c04000000a0: fa fa 00 01 fa fa fd fa fa fa fd fd fa fa fd fd
  0x1c04000000b0: fa fa fd fd fa fa fd fd fa fa fd fa fa fa fd fd
  0x1c04000000c0: fa fa fd fa fa fa fd fd fa fa fd fd fa fa fd fd
  0x1c04000000d0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x1c04000000e0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==20472==ABORTING

[tt4g]

I think it is a false positive for std::vector as described in the "False positives" section of https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.

We know about at least one kind of false positive container overflow reports. If a part of the application is built with asan and another part is not instrumented, and both parts use e.g. instrumented std::vector, asan may report non-existent container overflow. This happens because instrumented and non-instrumented bits of std::vector, inlined and not, are mixed during linking, so you end up with incompletely instrumented std::vector.

Solution: either build the entire application with asan or disable container overflow detection (ASAN_OPTIONS=detect_container_overflow=0)

[viniciusferrao]

Thanks @tt4g

For now I'm running the code with a wrapper to pass ASAN_OPTIONS=detect_container_overflow=0, as you suggested, on the shell. But let me check with you if I understood correctly: to avoid the workaround I should compile spdlog with sanitizers by myself. Since I'm using spdlog binaries from the distribution, the only way is to use the workaround, right?

[tt4g]

The description of google/sanitizers/wiki/AddressSanitizerContainerOverflow says so.

NOTE I don't use AddressSanitizer, so I may have overlooked something.