Skip to content

fix(services/gcs): encode object paths exactly once#7947

Draft
Xuanwo wants to merge 3 commits into
mainfrom
xuanwo/fix-gcs-encoding-with-reqsign-rc
Draft

fix(services/gcs): encode object paths exactly once#7947
Xuanwo wants to merge 3 commits into
mainfrom
xuanwo/fix-gcs-encoding-with-reqsign-rc

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 24, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes #7942.

This restores the fix for #7926 after #7927 was reverted by #7943. It depends on apache/opendal-reqsign#785, which is included in v0.20.3-rc.1.

Rationale for this change

GCS request builders own percent-encoding, but the writer currently pre-encodes logical object paths before passing them to those builders. Nested object names are therefore encoded twice and cannot be read back through the same OpenDAL path.

#7927 removed the writer-side encoding, but the reqsign implementation used at that time decoded and rebuilt caller-owned query URIs during signing. That caused real GCS behavior tests to fail and required the revert. The coordinated reqsign RC now defines the input URI as wire-ready and preserves it during header signing, so OpenDAL can encode the GCS path exactly once without the signer changing the request.

An A/B reproducer using the same GCS upload URI fails with the released reqsign crates (failed to apply signing request, URI replaced with /) and succeeds with v0.20.3-rc.1 while preserving the complete URI.

What changes are included in this PR?

  • Align all reqsign dependencies with the coordinated v0.20.3-rc.1 commit and pin it at repository scope so every Cargo workspace uses the same signing semantics.
  • Update the core and tracked Ruby lockfiles to the coordinated RC graph.
  • Pass logical GCS writer paths unchanged and percent-encode them once in request builders, including multipart initiation.
  • Add a signed-request regression test that asserts the wire URI is preserved and not double-encoded.
  • Extend the special-character behavior test with a nested path separator.

This PR is intentionally draft. Before it is ready to merge, reqsign 0.20.3 must be published to crates.io, the temporary [patch.crates-io] entries must be removed, and the real GCS behavior configurations must pass.

Are there any user-facing changes?

Yes. GCS writes to nested or reserved-character object names use the same logical object name as read, stat, and multipart operations. There is no public API or call-site change.

Verification

  • cargo check --workspace --all-targets --all-features --locked
  • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
  • Unit and doc tests for every reqsign-backed OpenDAL service
  • Locked dependency resolution from the core and Ruby workspaces, plus independent resolution from the .NET and OCaml workspaces
  • ./scripts/workspace.py cargo fmt -- --check

The full workspace doctest run reached opendal-service-foyer and then hit a local libjvm.dylib loader error unrelated to this change. The reqsign-backed service doctests pass.

AI Usage Statement

OpenAI Codex (GPT-5) was used to analyze the cross-repository signing contract, implement the change, and run validation. The resulting diff and test output were reviewed by the author.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: gcs behavior test failure

1 participant