Implement FEP-8fcf followers collection synchronization #2297
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds support for FEP-8fcf to synchronize followers collections across servers. This includes generating and processing Collection-Synchronization headers, computing partial follower digests, exposing a new /followers-sync REST endpoint, and scheduling async reconciliation jobs when mismatches are detected. New methods and traits are introduced for digest computation, header parsing, and reconciliation, with hooks for monitoring sync events. This improves data consistency and interoperability with other ActivityPub servers supporting FEP-8fcf.
github-actions
bot
added
[Feature] Collections
[Feature] REST API
[Status] In Progress
Docs
labels
Included a reference to FEP-8fcf (Followers collection synchronization across servers) in the list of supported FEPs to reflect updated protocol support.
Moved FEP-8fcf Collection-Synchronization header parsing and validation from Followers class to Http class, and generalized synchronization handling into the Collection trait. Removed the Followers_Sync trait and updated controllers to use the new process_collection_synchronization method. This refactor enables support for synchronizing additional collection types beyond followers and centralizes related logic for easier maintenance.
Changed the REST endpoint path from /followers-sync to /followers/sync in documentation, URL generation, and route registration to ensure consistency and clarity in the API.
Refactored code to remove redundant '\Activitypub\' class name prefixes when calling static methods within the same namespace. This improves code readability and consistency.
Moved Collection-Synchronization (FEP-8fcf) logic from REST controllers and trait to a dedicated handler class (Collection_Sync). Removed related code from trait-collection.php, class-http.php, and REST controllers, and registered the new handler in class-handler.php. This improves separation of concerns and centralizes collection sync logic.
There was a problem hiding this comment.
Pull Request Overview
This PR implements FEP-8fcf (Followers collection synchronization across servers) to improve data consistency and interoperability with other ActivityPub servers. The implementation adds Collection-Synchronization headers to outgoing activities, provides endpoints for partial follower collections, and includes async reconciliation when digest mismatches are detected.
Key changes:
- Adds Collection-Synchronization header generation and processing for FEP-8fcf protocol
- Implements partial follower digest computation using XOR'd SHA256 hashes
- Provides new
/followers-syncREST endpoint for synchronization - Includes async reconciliation scheduler for handling digest mismatches
Reviewed Changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| includes/scheduler/class-follower.php | New scheduler for handling async follower reconciliation |
| includes/rest/class-inbox-controller.php | Adds Collection trait for synchronization support |
| includes/rest/class-followers-controller.php | Implements new /followers/sync endpoint for partial collections |
| includes/handler/class-collection-sync.php | New handler for processing Collection-Synchronization headers |
| includes/collection/class-remote-actors.php | Fixes missing namespace import for Webfinger |
| includes/collection/class-followers.php | Adds FEP-8fcf digest computation and partial followers methods |
| includes/class-scheduler.php | Registers new Follower scheduler |
| includes/class-http.php | Adds Collection-Synchronization header to outgoing activities |
| includes/class-handler.php | Registers new Collection_Sync handler |
| docs/fep-8fcf-implementation.md | Documentation for FEP-8fcf implementation |
| FEDERATION.md | Updates supported FEPs list |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Introduces 'page', 'per_page', and 'order' parameters to the followers sync REST endpoint for improved pagination and sorting. Updates the controller to handle these parameters and enhances test coverage with extensive e2e tests for pagination, authority filtering, error handling, and response consistency. Also updates Playwright config to ensure proper rewrite structure for test environment.
Deleted the static get_authority method from Collection_Sync as it is no longer used in the codebase.
Corrects the regular expression in detect_collection_type to match '/followers/sync' instead of '/followers-sync', ensuring accurate detection of followers collections.
The webServer command now runs an additional command to set the WordPress permalink structure to '/postname' after starting the environment. This ensures tests run with the correct permalink settings.
Added a 'wp rewrite flush' command to the Playwright webServer setup to ensure rewrite rules are properly applied before running tests.
Removed the old 'fep-8fcf-implementation.md' and added a new, more comprehensive 'collection-synchronization.md' documentation. The new doc provides detailed information on the FEP-8fcf prototype, including implementation details, REST endpoints, privacy considerations, action hooks, compatibility, testing, and future enhancements.
Removed additional WordPress CLI commands from the webServer command, leaving only 'npm run env-start' for environment setup.
Moved test cases out of nested blocks in the 'Partial Followers Sync Endpoint' and 'Collection Response Format' suites for improved readability and consistency. No logic changes were made.
Updated the afterStart lifecycle script to flush rewrite rules after setting the permalink structure, ensuring changes take effect immediately.
The Collection trait was removed from the Inbox_Controller class as it is no longer used. This helps clean up the code and avoid unnecessary dependencies.
Co-authored-by: Copilot <[email protected]>
Updated HTTP message signature and draft signature classes to include the Collection-Synchronization header in the signed components when present. Also removed unnecessary early returns in Followers class digest computation methods.
Replaces the Follower scheduler with a new Collection_Sync scheduler for improved reconciliation of follower state per FEP-8fcf. Updates the documentation, handler, and signature verification to require the Collection-Synchronization header to be signed. Adds local following snapshot logic, updates action hooks, and enhances test coverage for signature headers.
Wraps the mapped followers and pending arrays with array_filter to remove null values, ensuring only valid actor IDs are returned. This prevents potential issues caused by invalid or errored actor lookups.
Moved the check for the 'collection-synchronization' header being part of the signature from the signature verification classes to the collection sync handler, as required by FEP. This streamlines the verification process and avoids duplicate checks.
Replaces usage of Collection\Followers with direct Followers import for generating the synchronization header in the HTTP class. This streamlines the code and improves clarity by using the imported Followers class directly.
Refactored FEP-8fcf collection synchronization logic by moving parse_collection_sync_header and xor_hex_strings from Http to Signature, and added compute_collection_digest to Signature. Updated Followers and Collection_Sync classes to use the new Signature methods for digest computation and header parsing. This centralizes collection sync logic and improves code organization.
Refactored the addition of the Collection-Synchronization header for Create activities (FEP-8fcf) from Http to Collection_Sync via a new http_request_args filter. This centralizes header logic, improves maintainability, and ensures the header is added consistently. Also adjusted the filter priority in Signature for better compatibility.
Included the 'user_id' parameter in relevant test cases within Test_Signature to ensure proper context for signature generation. Also updated a signature assertion to remove 'collection-synchronization' from the expected headers.
Refactored all relevant test files to use '/actors/' in REST API endpoints instead of '/users/' for consistency with the API specification. Also updated class-signature.php to use fully qualified PHP function names for better clarity and to avoid namespace issues.
Set 'http_request_args' filter in Signature to priority 0 and in Collection_Sync to -1 for better execution order. Update Collection_Sync to check 'HTTP_SIGNATURE_INPUT' header and always unslash signature input for improved FEP compliance.
Refactors maybe_add_headers to handle both JSON string and array bodies, ensuring the Collection-Synchronization header is only added for 'Create' activities. This improves robustness when processing different body formats.
Eliminates an unnecessary assignment of the 'body' parameter in the Collection_Sync class, as the value is already present in the arguments. This streamlines the code and avoids redundant operations.
Updated docblocks to consistently refer to `Create` activities in code comments, improving clarity and accuracy.
The get_authority method was removed from Http and reimplemented as get_url_authority in functions.php. All usages in collection sync handlers and scheduler were updated to use the new function, and related validation logic was refactored for consistency.
Replaces the local compute_digest_from_actor_urls method with Signature::compute_collection_digest and removes the redundant method. This centralizes digest logic in the Signature class for better maintainability.
Simplifies the validation of the followers collection ID by removing redundant normalization and default handling. Updates get_followers_collection_id to return WP_Error on failure and uses trailingslashit for comparison. Removes the unused normalize_collection_url method.
Replaces direct processing of followers collection sync with a generic 'activitypub_collection_sync' action hook, allowing for async processing and extensibility. Removes specific methods for detecting collection type and processing followers sync, simplifying the handler logic.
Updated the signature extraction logic in Collection_Sync to exclude the HTTP_SIGNATURE_INPUT header, now only considering HTTP_SIGNATURE and HTTP_AUTHORIZATION. This aligns with FEP requirements and simplifies the header handling.
Deleted a comment referencing the Collection-Synchronization header in the Create activities test, as it is no longer relevant to the test logic.
Updated the comment to specify that the 'http_request_args' filter runs ahead of all other filters, clarifying the intent for setting the signature early.
Removes redundant URL parameter check and the get_followers_collection_id method from Collection_Sync handler, simplifying followers collection retrieval. Updates the scheduler to generalize reconciliation scheduling for different collection types and removes unused filtering methods.
Renamed Followers::get_partial_followers to get_by_authority and updated its implementation to use get_url_authority for filtering. Updated all usages accordingly. In Following, renamed get_local_followers_snapshot to get_local_followers and changed the returned key from 'followers' to 'accepted' for clarity.
Replaces the call to Followers::get_partial_followers with Followers::get_by_authority when fetching partial followers filtered by authority. This ensures the correct method is used for retrieving followers based on authority.
Replaced get_by_authority with get_id_by_authority to improve follower filtering by authority using WP_Query and meta_query. Updated usages in Followers and Followers_Controller classes for consistency and better performance.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds support for FEP-8fcf to synchronize followers collections across servers. This includes generating and processing Collection-Synchronization headers, computing partial follower digests, exposing a new /followers-sync REST endpoint, and scheduling async reconciliation jobs when mismatches are detected. New methods and traits are introduced for digest computation, header parsing, and reconciliation, with hooks for monitoring sync events. This improves data consistency and interoperability with other ActivityPub servers supporting FEP-8fcf.
Proposed changes:
/followers/syncREST endpoint for synchronizationOther information:
Testing instructions:
Changelog entry
Changelog Entry Details
Significance
Type
Message
Added support for FEP-8fcf follower synchronization, improving data consistency across servers with new sync headers, digest checks, and reconciliation tasks.