-
Notifications
You must be signed in to change notification settings - Fork 606
Support and accelerate bitmap operations on more platforms #942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
And fast-path memcpy for single key
* Also increase coverage to show failure in current PR to handle uneven bitmap lengths on the two-key accelerated path
Some mixed results for benchmarks on different microarchitectures on some inputs. Secondary goal of this PR is to not regress performance, so still need to investigate that. |
Extremely hard to measure, especially wrt. binary input If some performance numbers seem now completely off from before on some important hardware you target, do report! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR refactors the bitmap operations SIMD logic to improve cross-platform compatibility, particularly enabling BITOP operations in Android VMs. The changes replace architecture-specific x86 SIMD code with modern .NET SIMD approaches using TensorPrimitives and generic vector operations that work across different platforms.
Key changes include:
- Refactored bitmap operation code to use TensorPrimitives and generic Vector types instead of x86-specific intrinsics
- Simplified test structure and improved environment variable testing for different SIMD configurations
- Added new binary operator abstractions for cross-platform bitwise operations
Reviewed Changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
File | Description |
---|---|
test/Garnet.test/TestProcess.cs | Refactored process management with cleaner constructor API and improved variable naming |
test/Garnet.test/GarnetBitmapTests.cs | Streamlined bitmap tests with environment-based SIMD testing and comprehensive multi-size test coverage |
libs/server/Storage/Session/MainStore/BitmapOps.cs | Updated bitmap operation storage logic to use new cross-platform SIMD manager |
libs/server/Resp/Bitmap/BitmapManagerBitOp.cs | Complete rewrite using TensorPrimitives and generic Vector types for cross-platform SIMD support |
libs/common/Numerics/IBinaryOperator.cs | New abstraction for cross-platform binary operations with Vector support |
benchmark/BDN.benchmark/Bitmap/*.cs | Added benchmark classes for performance testing of new bitmap operations |
Package files | Added System.Numerics.Tensors dependency for new SIMD implementation |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Co-authored-by: Copilot <[email protected]>
Last CI test failure is from cluster flakiness |
This PR refactors the SIMD logic used in bitmap operations to reduce duplication and simplify control flow. The goal is to improve maintainability and set the stage for consistent performance across architectures.
Vector512
path. This like others only lights up and is not DCE'd during JIT import if it is supported and hardware accelerated.ISimdVector<TSelf, T>
in the futureARM64 demo on Google Pixel 8a
Now we can do BITOPs on when running Garnet inside Android VMs!screen-20250316-174515.mp4
Follow-up work:
Refactor BITCOUNT logic similarly to be cross-platform(Platform independent bitcount operation #1034) and to follow "modern" .NET SIMD practices