fix(services/gcs): encode object paths exactly once#7947
Draft
Xuanwo wants to merge 3 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #7942.
This restores the fix for #7926 after #7927 was reverted by #7943. It depends on apache/opendal-reqsign#785, which is included in
v0.20.3-rc.1.Rationale for this change
GCS request builders own percent-encoding, but the writer currently pre-encodes logical object paths before passing them to those builders. Nested object names are therefore encoded twice and cannot be read back through the same OpenDAL path.
#7927 removed the writer-side encoding, but the reqsign implementation used at that time decoded and rebuilt caller-owned query URIs during signing. That caused real GCS behavior tests to fail and required the revert. The coordinated reqsign RC now defines the input URI as wire-ready and preserves it during header signing, so OpenDAL can encode the GCS path exactly once without the signer changing the request.
An A/B reproducer using the same GCS upload URI fails with the released reqsign crates (
failed to apply signing request, URI replaced with/) and succeeds withv0.20.3-rc.1while preserving the complete URI.What changes are included in this PR?
v0.20.3-rc.1commit and pin it at repository scope so every Cargo workspace uses the same signing semantics.This PR is intentionally draft. Before it is ready to merge, reqsign 0.20.3 must be published to crates.io, the temporary
[patch.crates-io]entries must be removed, and the real GCS behavior configurations must pass.Are there any user-facing changes?
Yes. GCS writes to nested or reserved-character object names use the same logical object name as read, stat, and multipart operations. There is no public API or call-site change.
Verification
cargo check --workspace --all-targets --all-features --lockedcargo clippy --workspace --all-targets --all-features --locked -- -D warnings./scripts/workspace.py cargo fmt -- --checkThe full workspace doctest run reached
opendal-service-foyerand then hit a locallibjvm.dylibloader error unrelated to this change. The reqsign-backed service doctests pass.AI Usage Statement
OpenAI Codex (GPT-5) was used to analyze the cross-repository signing contract, implement the change, and run validation. The resulting diff and test output were reviewed by the author.