Use memory addresses instead of creating static arrays in Rust template

[claudiomattera]

Static arrays will increase the size of the cartridge, because at runtime they will be initialized with the values used in the source (which is then ignored by the allocator).

If we start from an empty WASM-4 Rust project (the one created with w4 new --rust) and we add just these two lines:

let mut a: Vec<u8> = Vec::new();
a.push(1);

the cartridge size goes from 509 B to 27 KiB (actually to 64 KiB, because debug information are also retained, but those can be removed with wasm-gc and wasm-opt).

twiggy top -n 20 target/wasm32-unknown-unknown/release/*.wasm
 Shallow Bytes │ Shallow % │ Item
───────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
         20513 ┊    69.59% ┊ data[2]
          2216 ┊     7.52% ┊ "function names" subsection
          1158 ┊     3.93% ┊ core::fmt::Formatter::pad::h2c14c4abe79aa87d
           788 ┊     2.67% ┊ data[0]
           765 ┊     2.60% ┊ <buddy_alloc::non_threadsafe_alloc::NonThreadsafeAlloc as core::alloc::global::GlobalAlloc>::dealloc::hb690b3c45f61e9bb
           711 ┊     2.41% ┊ buddy_alloc::buddy_alloc::BuddyAlloc::new::h41b67823c01a5723
           402 ┊     1.36% ┊ buddy_alloc::buddy_alloc::BuddyAlloc::malloc::h860bb507167efed5
           398 ┊     1.35% ┊ <buddy_alloc::non_threadsafe_alloc::NonThreadsafeAlloc as core::alloc::global::GlobalAlloc>::alloc::he3ef24d488f93281
           197 ┊     0.67% ┊ alloc::raw_vec::finish_grow::h69f157004b2daf4d
           197 ┊     0.67% ┊ update
           190 ┊     0.64% ┊ memcpy
           187 ┊     0.63% ┊ memset
           129 ┊     0.44% ┊ buddy_alloc::fast_alloc::FastAlloc::new::h8da0dda8b25957a7
           127 ┊     0.43% ┊ core::result::unwrap_failed::hcbbd7f1ba3fa06cb
           120 ┊     0.41% ┊ std::panicking::rust_panic_with_hook::ha223473efd20893e
            92 ┊     0.31% ┊ core::option::expect_failed::h1b423ad3b3e89b13
            92 ┊     0.31% ┊ core::panicking::panic_str::hb981c042b527d2d8
            85 ┊     0.29% ┊ core::cell::RefCell<T>::borrow_mut::h2d4ba4c06e2fc5ad
            85 ┊     0.29% ┊ core::cell::RefCell<T>::borrow_mut::h4d31d5534c261c0c
            79 ┊     0.27% ┊ data[1]
           947 ┊     3.21% ┊ ... and 61 more.
         29478 ┊   100.00% ┊ Σ [81 Total Rows]

About 7 KiB of that seem to be due to buddy-alloc functions, but 20 KiB are allocated for the two static arrays FAST_HEAP and HEAP defined in alloc.rs (item data[2]).
If we change the values of FAST_HEAP_SIZE and HEAP_SIZE, the size of data[2] changes accordingly.
I think it is because static array must be initialized at runtime with their value specified in the source, which must be carried along in the cartridge.

If instead we bypass those arrays entirely and just use raw memory addresses, this does not happen.

I placed the two heap areas at the end of the memory (0x10000 - HEAP_SIZE - FAST_HEAP_SIZE), because if I put them right after the framebuffer I they just cause complete corruption (probably because the stack is growing from that side?).

Hopefully I am not missing out any obvious issue in using raw memory addresses.
I tested on my games and they seem to work the same.

[netlify]

✔️ Deploy Preview for wasm4 ready!

🔨 Explore the source changes: a5f9c23

🔍 Inspect the deploy log: https://app.netlify.com/sites/wasm4/deploys/6182b27249fc800007f2ec39

😎 Browse the preview: https://deploy-preview-138--wasm4.netlify.app

[aduros]

You're right that the 20 KB of zero bytes seems end up in the .wasm when using heap allocation. Luckily there's a flag to wasm-opt called --zero-filled-memory which signals that the imported memory will already be initialized to zero, so those data sections can be trimmed from the .wasm. Using that option shrinks the cart down by another 20 KB.

After investigating more, it looks like Rust also includes DWARF debug info even in release builds, there's another flag called --strip-dwarf to trim that too.

Using these flags shrinks minesweeper.wasm from 117K to 26K!!

wasm-opt minesweeper.wasm -o minesweeper-opt.wasm -Oz --strip-dwarf --strip-producers --zero-filled-memory

(--strip-producers strips ~150 bytes of "Built with Rust version X.Y.Z" metadata added by rustc from the wasm)

[claudiomattera]

I can only get it down to 41 KiB with that command, but it does indeed remove those data items from the cartridge.

Should wasm-opt be part of the build process then, instead of changing arrays to raw pointers?
In that case it should be documented.

[aduros]

Should wasm-opt be part of the build process then, instead of changing arrays to raw pointers?
In that case it should be documented.

Yeah, let's add some documentation about wasm-opt. Changing the allocator to use hard-coded addresses could introduce memory corruption that would be very hard for developers to debug.

[zetanumbers]

You cannot just define some random pointers and pretend there's a heap. It's just bad. There's #255 because of this.

Instead of this i would recommend to use MaybeUninit (assuming allocator doesn't require zeroed memory) to reduce cartridge size by eliminating a bunch of zeroes. It would place the heap into the bss section, which shouldn't be initiated thus no initialization zeroes. The only problem is llvm currently fills bss with zeroes too. Maybe we should fix this?