dealing with data too large for a single buffer

[alphastrata]

Connections
Link to the issues addressed by this PR, or dependent PRs in other repositories
discussion
thread on matrix

Description
The aim of this new example is to demonstrate taking a large input dataset, splitting it into chunks for the purpose of moving it onto the GPU, but then treating it as a single contiguous data structure once on the GPU.

Testing
Explain how this change is tested.

Checklist

• Run cargo fmt.
• Run cargo clippy. If applicable, add:
  • --target wasm32-unknown-unknown
  • --target wasm32-unknown-emscripten
• Run cargo xtask test to run tests.
• Add change to CHANGELOG.md. See simple instructions inside file.

[cwfitzgerald]

Sorry for the long wait time for a review!

Frankly as it exists right now, we cannot accept this example. While it physically shows one strategy for dealing with large data sets, after reading it, the user doesn't get a good idea of why that strategy should be used and what problems they are avoiding, compared to the more naive strategy of using larger and larger buffers. Through inline code comments and verbiage in the readme, the reader who has no idea about any of these topics (or even the details of memory allocation) should be able to understand why this is an effective strategy to utilize.

Some things I think it should touch on:

• Large buffers may fail to allocate due to fragmentation
• Growing/shrinking a dataset with a buffer system requires copying the entire buffer contents, whereas pagenated data just requires rebuilding a bind group.

I'm not going to close this, as I do think this can be transformed into something that would be great to have.

Added a few incidental comments.

[cwfitzgerald]

Empty file? This example definitely needs tests

[cwfitzgerald]

These defaults should work everywhere, they're the minimum required by webgpu.

[alphastrata]

Sorry for the long wait time for a review!

Frankly as it exists right now, we cannot accept this example. While it physically shows one strategy for dealing with large data sets, after reading it, the user doesn't get a good idea of why that strategy should be used and what problems they are avoiding, compared to the more naive strategy of using larger and larger buffers. Through inline code comments and verbiage in the readme, the reader who has no idea about any of these topics (or even the details of memory allocation) should be able to understand why this is an effective strategy to utilize.

Some things I think it should touch on:

• Large buffers may fail to allocate due to fragmentation
• Growing/shrinking a dataset with a buffer system requires copying the entire buffer contents, whereas pagenated data just requires rebuilding a bind group.

I'm not going to close this, as I do think this can be transformed into something that would be great to have.

Added a few incidental comments.

Cheers, I'll keep working on it.