feat(single-node): add memory allocation

[fuyufjh]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Follow up of #19477. This makes it work in single-node mode.

Checklist

• I have written necessary rustdoc comments.
• I have added necessary unit tests and integration tests.
• I have added test labels as necessary.
• I have added fuzzing tests or opened an issue to track them.
• My PR contains breaking changes.
• My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
• My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

• My PR needs documentation updates.

Release note

[hzxa21]

I had a discussion of @Li0k recently on the memory allocation between CN and compator in standalone.
Let's say CN_MEM = x * Total_MEM, Compactor_MEM = y * Total_MEM, FE_MEM = z * Total_MEM, the choices are

1. x + y + z = 1.

   • Pros: more likely to avoid OOM.
   • Cons: less efficient on memory usage because when loads on compaction/FE is not large, the memory will be wasted and cannot used by CN.

2. x + y + z > 1

   • Pros: More efficient on memory usage because CN operator cache can use the idle memory when compactor/FE is not fully loaded.
   • Cons: More vulnerable to OOM because CN operator cache eviction is lazy.

I am more leaning towards 2 because it fits more to the CN dynamic operator cache design. For example, we can have x = y = z = 0.8 as well as the gradient allocation. WDYT?

[fuyufjh]

CN operator cache is special - keep in mind that it's controlled according to the process-level jemalloc statistics. (See memory_manager_target_bytes in next line)

Here the 3 options

• frontend_opts.frontend_total_memory_bytes
• compactor_opts.compactor_total_memory_bytes
• compute_opts.total_memory_bytes

Mostly decides decides the storage memory, including uploading buffer and meta & block cache:

• frontend_opts.frontend_total_memory_bytes (code here)
  • batch_memory_limit
• compactor_opts.compactor_total_memory_bytes (code here)
  • meta_cache_capacity_bytes
  • compactor_memory_limit_bytes
• compute_opts.total_memory_bytes decides (code here)
  • block_cache_capacity_mb
  • meta_cache_capacity_mb
  • shared_buffer_capacity_mb
  • (embedded) compactor_memory_limit_mb

As you are more familiar with storage, I'll follow your decisions.

Besides, it doesn't count the Meta's memory usage (because there is no such as option), so actually it's already > 1.

[Li0k]

According to the example above

x = y = z = 0.8

Total memory is well over 1.0 (0.8 * 3 = 2.4)
My concern is that when we use “excessive” configurations, it will cause the CN's memory to be evicted faster (according to jemalloc statistics), which will severely weaken the CN's capabilities.

[Li0k]

I'm also in favor of option 2, but it seems that the limit should not be exceeded. For example, it is only allowed to exceed 0.5 or less, in order to guarantee the capacity of the CN.

• The memory footprint of the compactor is not a long term “cache”, so it is freed when it is used up, and it can be more “over-occupied”.
• The FE footprint seems to come from batch query? I'm not sure it's released in time cc @fuyufjh @hzxa21

[Li0k]

LGTM