Alternative to the GatherScatterer functions?

[BlueCyro]

Hi, I've been taking some pretty heavy inspiration from this library for use in a funny little software rasterizer I'm working on. I saw the shiny-looking GatherScatterer class and the methods in Vector2/3/4Wide where it's used. I started to use it only to see your comment about how gather/scatter should ideally be replaced by vectorized transposes. I was wondering what that might look like for say, Vector3Wide for example if I wanted to accomplish similar re-ordering of the values.

I'm still learning how best to utilize SIMD to accelerate math-heavy workloads so forgive me if some of my terminology or use-cases are a a little inaccurate or misguided :p

[RossNordby]

A relevant, but still messy-and-something-I'd-like-to-improve, implementation is over here.

Some notes:

1. Actual gather instructions exist on a lot of newer hardware, but are somewhat restrictive in their usage and are often not significantly better than a scalar implementation.
2. While gather operations will typically run into cache miss induced latency or bandwidth issues before raw instruction throughput becomes a bottleneck, it is worth noting that there are limits on how many load instructions a core can issue. Loading more data at once helps.
3. These implementations make use of the fact that body data is laid out into chunks of two cache lines per body. All solver-relevant data for a body can be loaded by requesting 128 contiguous bytes with vector loads.
4. Given the way the solver works, there's no way to avoid noncontiguous loads with respect to different bodies at some point in a general simulation. Depending on the size of the simulation, it's likely that much of the data you'd like to load is not in the most core-local cache, and you'll need to go grab it from elsewhere.
5. Cache misses take a long time and the core can't do anything that relies on the loaded information until it's available. But... if you need to load a bunch of independent stall-inducing memory, modern out-of-order execution cores are quite good at doing nothing in parallel! Dispatching loads for all independent lanes within a bundle works well for this.

[BlueCyro]

Yeah! I was actually taking a look at the GatherState function quite a lot. I kind of mushed together a crummy Transpose for Vector2/3/4Wide to unpack the struct all at once, but it actually ended up slower since I think I'm missing a few pieces of the puzzle still...

How're you ensuring correct alignment in these performance-sensitive areas? Right now I've just got some dumb structs that wrap an aligned pointer from NativeMemory.AlignedAlloc, but I have a feeling that's not quite enough if I want to read packed triangles from memory efficiently and subsequently hammer an aligned pixel buffer with a crapload of load/store operations.

Memory alignment I think is the part that's killing me here, but unfortunately all of the examples I can find are either in really esoteric C++ questions on stackoverflow, or very old and now no longer apply.

Thanks for the response by the way!

[RossNordby]

Any memory buffer used in these areas is aligned (using something roughly equivalent to NativeMemory.AlignedAlloc), and the size of the structs stored in sequence within those buffers is chosen to maintain alignment. The solver-relevant body data is 128 bytes, for example, so if the start pointer is aligned, so will all structs following it.