Add process_info binary_full that provides more details

[michalmuskala]

In particular, this includes the full original binary and all the references. Gathering this information requires full heap traversal, so is fairly expensive - that's why we keep both the existing binary and this new API.

Example:

1> A = <<0:(1024*8)>>.
2> <<B:550/bitstring,D:550/bits,E:550/bits,C/bitstring>> = A.
3> erlang:process_info(self(),[binary_full]).
[{binary_full,[{126812357133856,1024,1,
           <<0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,...>>,
           [{1100,1650},{550,1100},{1650,8192},{0,550},{0,8192}]}]}]

Original draft implementation by @garazdawi in garazdawi@d5a2c6f.

[github-actions]

CT Test Results

    3 files    135 suites   49m 39s ⏱️
1 651 tests 1 594 ✅ 57 💤 0 ❌
2 287 runs  2 211 ✅ 76 💤 0 ❌

Results for commit 39c06bb.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

• Complete CT logs (Download Logs)
• HTML Documentation (Download HTML Docs)
• Windows Installer

// Erlang/OTP Github Action Bot

[jhogberg]

Thanks for the PR! I've left some comments below.

[jhogberg]

Nitpick:

Suggested change

	static void gather_binaries(DebugBinary *bins, Uint count, Eterm *start, Eterm *stop) {
	static void gather_binaries(DebugBinary *bins, Uint count, const Eterm *start, const Eterm *stop) {

[jhogberg]

This looks a bit complex, there's only one case with special treatment in either switch and there's fall-through on top of it, how about something like (untested)?

Suggested change

	switch (primary_tag(val)) {
	case TAG_PRIMARY_IMMED1:
	case TAG_PRIMARY_LIST:
	case TAG_PRIMARY_BOXED:
	break;
	case TAG_PRIMARY_HEADER:
	if (header_is_transparent(val)) {
	continue;
	}
	switch (thing_subtag(val)) {
	case SUB_BITS_SUBTAG:
	{
	const ErlSubBits *sb = (ErlSubBits*)(tp-1);
	const BinRef *underlying = (BinRef*)boxed_val(sb->orig);
	if (thing_subtag(underlying->thing_word) != HEAP_BITS_SUBTAG) {
	for (Uint i = 0; i < count; i++) {
	DebugBinary* b = &bins[i];
	if (b->bin_ref == underlying) {
	WSTACK_PUSH2(b->ws.ws, sb->start, sb->end);
	break;
	}
	}
	}
	}
	ERTS_FALLTHROUGH();
	default:
	{
	tp += header_arity(val);
	}
	break;
	}
	break;
	}
	if (primary_tag(val) == TAG_PRIMARY_HEADER &&
	!header_is_transparent(val)) {
	if (thing_subtag(val) == SUB_BITS_SUBTAG) {
	const ErlSubBits *sb = (ErlSubBits*)(tp-1);
	const BinRef *underlying = (BinRef*)boxed_val(sb->orig);
	if (thing_subtag(underlying->thing_word) != HEAP_BITS_SUBTAG) {
	for (Uint i = 0; i < count; i++) {
	DebugBinary* b = &bins[i];
	if (b->bin_ref == underlying) {
	WSTACK_PUSH2(b->ws.ws, sb->start, sb->end);
	continue;
	}
	}
	}
	}
	tp += header_arity(val);
	}

[jhogberg]

mbuf and msg_frag need to be searched separately, as you can have heap fragments without messages (e.g. to satisfy allocations when GC is temporarily disabled). Both have next members that you should follow, and the latter should keep using erts_message_to_heap_frag like you do now.

[jhogberg]

Did you intend a copy here? (Same issue in the other loop further down)

Suggested change

	DebugBinary b = binariesp[i];
	const DebugBinary *b = &binariesp[i];

[jhogberg]

If I read this correctly, bits[0] is the start offset of the referenced region and bits[1] is the end offset. How about naming bits something like range_offsets or slice_bounds instead?

[jhogberg]

Nitpick: prefer NBITS instead of * 8

Suggested change

	bitstring = erts_wrap_refc_bitstring(
	&hfact->off_heap->first,
	&hfact->off_heap->overhead,
	&hp, b.bin_ref->val, (byte*)b.bin_ref->val->orig_bytes,
	0, b.bin_ref->val->orig_size * 8);
	bitstring = erts_wrap_refc_bitstring(
	&hfact->off_heap->first,
	&hfact->off_heap->overhead,
	&hp, b.bin_ref->val,
	(byte*)b.bin_ref->val->orig_bytes,
	0,
	NBITS(b.bin_ref->val->orig_size));

[jhogberg]

Nitpick:

Suggested change

	if (OLD_HEAP(rp))
	gather_binaries(binariesp, binaries_count, OLD_HEAP(rp), OLD_HTOP(rp) /*OLD_HEND(p)*/);
	if (OLD_HEAP(rp)) {
	gather_binaries(binariesp, binaries_count, OLD_HEAP(rp), OLD_HTOP(rp) /*OLD_HEND(p)*/);
	}