[WIP] compilesymbolunits to keep track of instruction size, static ram usage and rodata

[PetervdPerk]

I'm happily using bloaty to keep track of the flash usage in my microcontroller projects, however Bloaty is only limited to keep track flash usage using File Size and VM size, whereas for microcontrollers you also want to keep track of static ram/heap usage. Furthermore it's also nice to keep track off read-only data (RODATA) to utilize the Harvard CPU architecture mostly used in microcontrollers to save on SRAM usage.

This is an experimental that branch implements the compilesymbolunits datasource output option, which adds extra information from the symbol table into the following columns:

• FUNC - Size all functions (program size) for compile unit
• DATA - Pre-allocated data (initialized & unitialized)
• RODATA - Read-only (const) data

Example output of bloaty

    FILE SIZE        VM SIZE        FUNC SIZE       DATA SIZE     RODATA SIZE   
 --------------  --------------  --------------  --------------  -------------- 
  37.2%  5.37Mi   28.1%  2.04Mi   38.7%   890Ki   15.6%   279Ki   18.7%   417Ki     [123 Others]
   8.3%  1.19Mi    4.5%   333Ki    9.8%   225Ki    0.0%       0    0.0%       0     bloaty/src/bloaty.cc
   1.0%   148Ki   14.6%  1.06Mi    1.4%  32.4Ki   57.6%  1.01Mi    0.0%       0     bloaty/third_party/capstone/arch/M68K/M68KDisassembler.c
   6.6%   978Ki   11.8%   878Ki    0.4%  8.56Ki    9.2%   165Ki   29.1%   647Ki     bloaty/third_party/capstone/arch/X86/X86Mapping.c
   5.3%   785Ki    7.6%   563Ki   10.5%   241Ki   10.0%   180Ki    2.4%  53.2Ki     bloaty/third_party/capstone/arch/ARM/ARMDisassembler.c
   4.7%   695Ki    2.1%   157Ki    4.4%   101Ki    0.0%       0    0.0%       0     bloaty/src/dwarf.cc
   3.6%   529Ki    5.9%   441Ki    0.8%  19.4Ki    0.0%       0   18.8%   418Ki     bloaty/third_party/capstone/arch/X86/X86DisassemblerDecoder.c
   3.3%   487Ki    1.0%  75.2Ki    2.2%  51.5Ki    0.0%     141    0.0%       0     bloaty/third_party/re2/re2/dfa.cc
   3.1%   459Ki    1.1%  83.3Ki    2.4%  55.6Ki    0.0%     140    0.0%       0     bloaty/third_party/re2/re2/regexp.cc
   3.0%   447Ki    0.7%  53.9Ki    1.6%  36.5Ki    0.0%      36    0.0%       4     bloaty/third_party/re2/re2/re2.cc
   2.8%   413Ki    3.8%   279Ki    0.0%     789    2.2%  39.0Ki    8.1%   180Ki     bloaty/third_party/capstone/arch/SystemZ/SystemZMapping.c
   2.7%   398Ki    1.6%   119Ki    1.3%  29.7Ki    0.8%  13.9Ki    3.2%  70.7Ki     bloaty/third_party/capstone/arch/X86/X86ATTInstPrinter.c
   2.6%   377Ki    2.0%   148Ki    5.3%   122Ki    0.0%       0    0.0%       0     bloaty/third_party/demumble/third_party/libcxxabi/cxa_demangle.cpp
   2.5%   376Ki    0.9%  65.7Ki    1.9%  44.8Ki    0.0%       8    0.0%       0     bloaty/third_party/re2/re2/prog.cc
   2.3%   332Ki    3.0%   219Ki    7.4%   170Ki    0.0%       0    1.1%  24.9Ki     bloaty/third_party/capstone/arch/AArch64/AArch64InstPrinter.c
   2.2%   327Ki    0.7%  48.9Ki    1.4%  33.2Ki    0.0%     148    0.0%       0     bloaty/third_party/re2/re2/compile.cc
   2.0%   298Ki    2.2%   163Ki    4.2%  96.7Ki    0.6%  11.2Ki    2.1%  46.2Ki     bloaty/third_party/capstone/arch/AArch64/AArch64Disassembler.c
   1.9%   281Ki    3.2%   237Ki    0.1%  2.41Ki    2.2%  38.7Ki    8.0%   178Ki     bloaty/third_party/capstone/arch/AArch64/AArch64Mapping.c
   1.8%   260Ki    2.9%   213Ki    0.1%  1.89Ki    1.4%  26.0Ki    7.6%   169Ki     bloaty/third_party/capstone/arch/ARM/ARMMapping.c
   1.7%   247Ki    0.7%  50.8Ki    1.7%  39.9Ki    0.0%      60    0.0%       0     bloaty/third_party/re2/re2/parse.cc
   1.5%   220Ki    1.7%   126Ki    4.2%  96.3Ki    0.0%       0    0.9%  21.0Ki     bloaty/third_party/capstone/arch/PowerPC/PPCInstPrinter.c
 100.0%  14.4Mi  100.0%  7.27Mi  100.0%  2.25Mi  100.0%  1.75Mi  100.0%  2.18Mi     TOTAL

Known issues

• It's feels quite hacky/inefficient to do it this way, mainly the copy actions in ComputeRollup and https://github.com/PetervdPerk/bloaty/blob/compilesymbolunits/src/elf.cc#L1389
• Introduced unnecessary overhead/compute when running other options then compilesymbolunits
• Current design uses the DualMap class, through hacking I've added the func_map, data_map and rodata_map RangeMaps, maybe there's a cleaner way to do this
• C++ doesn't export all DWARF symbols for data symbols e.g. _ZTSN6bloaty5ErrorE is no where to be found.
• Input parsing no fully implemented ShowSection is always true

[google-cla]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[haberman]

I am curious, does a breakdown by section give you what you need? eg.

$ ./bloaty bloaty -d compileunits,sections --domain=vm -n 5
     VM SIZE    
 -------------- 
  36.8%   198Ki    [59 Others]
  22.4%   120Ki    ../src/bloaty.cc
    75.5%  91.3Ki    .text
    10.1%  12.2Ki    .rodata
     8.4%  10.1Ki    .eh_frame
     4.0%  4.78Ki    .rela.plt
     1.1%  1.29Ki    [11 Others]
     1.0%  1.22Ki    .eh_frame_hdr
  21.8%   117Ki    ../third_party/demumble/third_party/libcxxabi/cxa_demangle.cpp
    87.5%   102Ki    .text
     9.6%  11.3Ki    .rodata
     2.5%  2.95Ki    .eh_frame
     0.3%     320    .eh_frame_hdr
     0.0%      24    .dynsym
     0.0%      15    [7 Others]
   8.1%  43.8Ki    ../src/dwarf.cc
    85.2%  37.4Ki    .text
    10.2%  4.46Ki    .eh_frame
     3.1%  1.34Ki    .rodata
     1.5%     688    .eh_frame_hdr
   5.4%  29.4Ki    ../third_party/abseil-cpp/absl/strings/charconv.cc
    75.5%  22.2Ki    .rodata
    17.5%  5.15Ki    .text
     2.0%     595    .dynstr
     1.9%     576    .dynsym
     1.9%     576    .rela.dyn
     1.2%     352    [8 Others]
   5.4%  29.0Ki    ../src/macho.cc
    65.1%  18.9Ki    .text
    25.7%  7.46Ki    .rodata
     8.4%  2.43Ki    .eh_frame
     0.8%     248    .eh_frame_hdr
 100.0%   539Ki    TOTAL

You can use custom data sources to bucket section names into broader categories like "FUNC", "DATA", "RODATA" according to whatever logic you want.

If you want to see this breakdown in columns instead of rows, I have a WIP change for this that implements a --pivot option, so you could write:

./bloaty bloaty -d sections,compileunits --domain=vm -n 5 --pivot
      .text          .rodata        .eh_frame       [Others]     .gcc_except_table      .dynstr   
 --------------  --------------  --------------  --------------  -----------------  --------------
        116Ki          1.62Ki          14.6Ki               0               0                 15     [Others]
        101Ki          12.6Ki          11.3Ki               0               0                400     ../src/bloaty.cc
        102Ki          11.3Ki          2.95Ki               0               0                  0     ../third_party/demumble/third_party/libcxxabi/cxa_demangle.cpp
       36.3Ki          1.33Ki          4.43Ki               0               0                  0     ../src/dwarf.cc
       22.9Ki               0               0               0               0                  0     ../src/elf.cc
            0          22.2Ki               0               0               0                595     ../third_party/abseil-cpp/absl/strings/charconv.cc
       20.6Ki               0               0               0               0                  0     ../src/demangle.cc
            0               0               0               0          15.5Ki                  0     [section .gcc_except_table]
            0               0               0               0               0             8.34Ki     [section .dynstr]
            0          7.46Ki               0               0               0                  0     ../src/macho.cc
            0               0          3.71Ki               0               0                602     src/bloaty.pb.cc
            0               0          3.52Ki               0               0                  0     [section .eh_frame]
            0               0               0               0               0                 75     ../third_party/abseil-cpp/absl/base/internal/raw_logging.cc
            0               0               0               0               0                  0     [ELF Headers]

It needs some work though. Would this satisfy your needs? With a custom data source, you could get output a lot like your example.

[PetervdPerk]

Indeed a section breakdown provides the information what I need but it tends to be hard to read and filter out the outliers in .rodata, .data or .bss sections.

If you want to see this breakdown in columns instead of rows, I have a WIP change for this that implements a --pivot option, so you could write:

Yes the pivot option would be pretty much solve the problem I've got, and if you even could sort on these rows (e.g. sort by .rodata) that would be perfect.