updates to v2 #11
Open
updates to v2 #11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merges v0.2 branch introducing significant improvements: - Add AsyncRead/AsyncWrite support for Tag and Stream (utils feature) - Add Worker::connect_addr_vec for raw address connections - Update UCX to 1.18.1 for latest API compatibility - Introduce RequestParam builder pattern for request handling - Add stream wrapper types (WriteStream, ReadStream, etc.) - Migrate to Rust 2021 edition - Update dependencies and fix various bugs Signed-off-by: Ryan Olson <[email protected]>
Add intelligent build system that prefers system UCX installation: - Try pkg-config first with version constraints (>= 1.19, < 2.0) - Use system installation if version requirements are met - Fall back to building from source if not found or incompatible - Support UCX_NO_PKG_CONFIG env var to force source build Benefits: - Faster builds when system UCX is available - Better integration with system package managers - Still maintains portability via source fallback Add pkg-config 0.3 as build dependency for UCX detection. Signed-off-by: Ryan Olson <[email protected]>
Update UCX submodule from 1.18.1 to v1.19.0 release tag for latest stable version with bug fixes and improvements. Signed-off-by: Ryan Olson <[email protected]>
When building UCX from source, the libraries are built as static archives that depend on each other. Must link all libraries explicitly: - libucp: main UCP API - libuct: UCX transport layer - libucs: UCX services and utilities - libucm: UCX memory management This fixes linker errors in CI where pkg-config is not available and the build falls back to building from source. Fixes undefined symbol errors for functions like ucp_config_read, ucp_init_version, ucp_worker_create, etc. Signed-off-by: Ryan Olson <[email protected]>
Refactor WorkerAddress to own its data using bytes::Bytes instead of holding a raw pointer with lifetime dependency on Worker. This makes WorkerAddress cloneable, 'static, and easier to work with. Changes: - Add bytes dependency to Cargo.toml - Remove lifetime parameter from WorkerAddress (now 'static) - Store address data in Bytes instead of raw pointer - Worker::address() now copies UCX address data and releases it immediately - Add Clone derive to WorkerAddress - Add constructors: from_bytes(), From<Bytes>, From<Vec<u8>> - Add as_bytes() accessor method Benefits: - WorkerAddress can be cloned and sent across channels/threads - Address data is owned and safe (no raw pointers) - Better support for out-of-band address exchange - Simpler lifetime management Tests: - Add worker_address_clone_and_from test - Add worker_address_connect_ping_pong test demonstrating: * Two workers exchanging addresses via channels * Connecting using WorkerAddress * Bidirectional communication (ping message) All existing tests pass (10 total tests passing). Signed-off-by: Ryan Olson <[email protected]>
Signed-off-by: Ryan Olson <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I updated the V2 PR and made a few build.rs changes to use locally installed ucx picked up from pkg-config.
I also made WorkerAddress be detachable form the Worker so it can be communicated between workers.
I can't for the life of me figure out why CI is broken.