fix(remote): never fail a build on remote config, restore legacy prefixes

[jleni]

Follow-up hardening for the OpenDAL remote (#593), from a review of that change after it merged.

Why this is worth landing soon

Three of these break setups that worked before #593, and the first one breaks builds rather than just cache hits.

A misconfigured remote failed every compiler invocation

load_remote_config became fallible in #593, Config::load propagates it with ?, and so does run_wrapper_mode. A remote config error therefore meant kache exited non-zero and rustc never ran.

Two configs that the pre-OpenDAL loader accepted started hard-failing on that path:

• prefix = "team/" (or any leading/trailing/duplicated slash)
• KACHE_S3_PREFIX="", which used to mean "store at the bucket root"

The reason is now recorded in Config::remote_error, the build continues local-only, and commands that exist to talk to the remote surface it through require_remote(). kache status reports MISCONFIGURED with the reason instead of claiming no remote was configured.

Legacy prefixes are normalized instead of rejected

Non-canonical prefixes normalize and log a warning. Only backslashes and . / .. segments stay errors, because there is no safe normalization for those.

Normalization moves objects: prefix = "team/" previously wrote team//v3/... and now writes team/v3/..., so the remote repopulates once. That is a deliberate trade against the alternatives of failing the build or disabling the remote entirely. Cache keys are content hashes and the layout layer verifies blake3 per file, so sharing a namespace across two spellings of the same prefix cannot serve a wrong artifact.

An empty prefix addresses the bucket root, which needed join_remote_key so keys do not gain a leading /. Without that, the config parsed but every operation failed key validation.

An empty KACHE_S3_BUCKET selected a different bucket

.filter(|v| !v.is_empty()) made a present-but-empty override fall through to the file-configured bucket, so a job that neutralized its remote silently got the one from the config file. It now disables the remote.

The S3 client had no connect or request timeout

Only pool_idle_timeout was set, which just reaps idle pooled connections. An endpoint that accepted a connection and then went silent hung head/get/put forever, and RetryLayer could not fire because no error was ever produced. On the wrapper path that looks like a hung build.

Restored a 3.1s connect timeout, matching SDK_DEFAULT_CONNECT_TIMEOUT from the aws-config this replaced, plus a 30s read-inactivity deadline. Not a total-request timeout, which would cap large transfers on slow links.

Transport and filesystem

• credential_process is re-lexed with shlex-style rules and executed directly, replacing sh -c / cmd.exe /C. reqsign parses the configured command with command.split_whitespace() and does not strip quotes, so quote characters arrive inside the tokens and the quoting does have to be reapplied. Doing it with a shell also expands globs, $(...) and %VAR%, treats & | ^ as operators, and under cmd.exe does not regroup single quotes at all. Known limitation: reqsign's split has already collapsed runs of whitespace inside quoted arguments, and no amount of re-lexing here can recover that.
• A credential_process child now receives the selected AWS_PROFILE. ProfileSelectingEnv only redirects reqsign's own in-process reads, but the helper is a separate process inheriting kache's environment, so it saw the ambient profile. A helper that shells out to AWS tooling could return credentials for a different account than profile asked for, silently. The child is spawned directly with AWS_PROFILE set on it alone; the process environment is still never mutated. This removed the last use of reqsign-command-execute-tokio, so that direct dependency is gone.
• Filesystem writes verify the resolved target stays inside the configured root. OpenDAL's fs service canonicalizes only the root, once at build time, so a symlink planted below it redirected writes out of the cache. Best effort, with a TOCTOU window that a hermetic fix would need openat2(RESOLVE_BENEATH) to close.
• get rejects a body shorter than its advertised length. Over HTTP the transport catches this first; the check covers a backend that ends the stream cleanly, such as the filesystem stat-then-read race.
• LIST gained a total deadline and entry/byte caps. The existing per-entry timeout only caught a stalled lister, not one trickling fresh entries forever.
• Keys reject control characters, and trailing dots or spaces on filesystem remotes, which Windows strips and which would make two distinct keys collide.
• atomic_write_dir is rejected when inside the object tree (staging files would list as cached objects) or on another filesystem (rename fails with EXDEV on every write). The EXDEV check runs at backend creation, not in Config::load, so it never stats an unavailable mount on the wrapper path.

Smaller items

• The JSON report carries network.configured_backend, so a consumer can tell whether the HTTP-named metrics describe a network or a local mount. It reflects the backend configured when the report is generated, not per transfer, and says so.
• default-features = false on the remaining OpenDAL crates. The resolved graph is unchanged; the manifest now matches the stated policy.
• sync --workspace validates the workspace before resolving credentials, so it no longer prompts for SSO and then reports there are no members.
• Docs updated for the new prefix rules, the staging containment rule, and the shared-directory trust boundary.

Deliberately not done

deny_unknown_fields on [cache.remote] looked attractive but is a trap. Serde fails the whole FileConfig parse on an unknown key, so a single typo takes every other setting with it. Measured with key_salt = "from-file" in [cache] and a typo'd bukcet under [cache.remote]:

key_salt     = None      <- file value lost, so the cache key shifts
remote       = None
remote_error = None      <- not even a reason to report

Silently rebuilding the world because of one typo is a worse outcome than ignoring the typo, and the config does not even end up with something to show in kache status. Reporting unknown keys properly needs remote parsing isolated from the rest of the config.

Validation

• cargo test -q -p kache --locked (1333 unit tests plus all integration suites)
• cargo clippy -q -p kache --all-targets --locked -- -D warnings
• cargo fmt --all -- --check
• cargo deny check bans licenses sources

Every regression fix has a test that fails without it. The symlink containment test was checked red by disabling the guard, so it is not a vacuous gate.