ByteBuffer: Investigate optimal allocation sizes (probably just under a power of two)

[weissi]

Currently, ByteBuffer always allocates in powers of 2 which makes sense. But many allocators have a small amount of overhead which means the actual space reservation might be even bigger, possibly another power of two, often unnecessarily.

For example the following could happen:

• The user needs to write 9,000 bytes and asks for allocator.buffer(capacity: 9_000)
• ByteBuffer gets a malloc(16_384) (power of 2)
• libc's allocator has (for the example's sake) 16 bytes of overhead storing some metadata
• This means we now require 16384 (for the malloc) + 16 bytes (allocator overhead) = 16394 which isn't a power of two

This could yield to an extra page being mapped for nothing (we only needed 9000 bytes after all). NIO's ByteBuffer should probably allocate just under the next power of two to leave some slack for the allocator in case it needs it. Needs investigation.

This becomes even more important if/when ByteBuffer is able to use tail allocations.

[hassila]

See some note to consider in the related case:

#2723 (comment)

[weissi]

Similar to #2770

[hassila]

Maybe we could be allowed to optionally provide a malloc size strategy that could default to the current power-of-two behavior but let one provide the result from the Darwin function (or some other heuristic of we know we want an exact size as in the other case I linked to, or if we know optimal sizes for the current allocator in use eg. Jemalloc)?

This would give us the ability to influence overhead. The size returned by such a closure would need to be bigger or equal to the requested size of course.

[Lukasa]

I think this wraps into a broader question around custom allocator patterns. There's been some interest in us actually making ByteBufferAllocator into a more meaningful type, and some of this will intersect with that.