[alukach]

source-coop exec -- COMMAND runs any S3 tool with the data proxy as its endpoint, as aws-vault exec does for AWS. Signed in, the command gets your credentials, so it needs no profile. That matters for tools that can't use the credential_process profile or ignore its endpoint: GDAL before 3.12 (most QGIS, rasterio and geopandas installs), Rust's object_store (obstore, Polars) and s5cmd. Signed out, the command still runs, unsigned, so anyone can read public products without signing in.

# signed in: your credentials, the proxy as the endpoint
source-coop exec -- aws s3 ls s3://my-org/my-data/
source-coop exec -- aws s3 cp s3://my-org/my-data/data.parquet .
source-coop exec -- s5cmd cp 's3://my-org/my-data/*.parquet' ./local/
source-coop exec -- gdalinfo /vsis3/my-org/my-data/image.tif
source-coop exec -- python script.py

# signed out: public data, unsigned (the AWS CLI and s5cmd need the flag)
source-coop exec -- aws --no-sign-request s3 ls s3://cholmes/admin-boundaries/
source-coop exec -- ogrinfo -so /vsis3/cholmes/admin-boundaries/countries.parquet

# another role or environment, as for creds
source-coop exec --role-arn reader-role --proxy-url https://data.staging.source.coop -- aws s3 ls s3://my-org/

How it behaves

• Everything after -- is the command. On Unix the CLI becomes the command (execve), so its exit code and signals (Ctrl+C) reach the shell directly. On Windows it waits and passes the exit code on. A command that can't be started is Error: Couldn't run …, exit 1. No command at all is a usage error, exit 2.
• The proxy as the S3 endpoint, under each tool's name for it: AWS_ENDPOINT_URL_S3 (AWS CLI v2.13+, SDKs), S3_ENDPOINT_URL (s5cmd), AWS_ENDPOINT (object_store before 0.13.2, plus AWS_ALLOW_HTTP for a plain-HTTP proxy), and AWS_S3_ENDPOINT, AWS_HTTPS, AWS_VIRTUAL_HOSTING=FALSE (GDAL /vsis3/). The global AWS_ENDPOINT_URL isn't set, so a script's calls to other AWS services keep AWS's endpoints.
• A region: AWS_REGION and AWS_DEFAULT_REGION are set to us-west-2, the same as service-account usage gives, only if neither is already set to a non-empty value.
• Always removed: AWS_PROFILE, AWS_DEFAULT_PROFILE, AWS_IGNORE_CONFIGURED_ENDPOINT_URLS and AWS_ENDPOINT_URL, so neither a profile, that switch, nor a general endpoint (LocalStack, say) can send S3 requests anywhere but the proxy. object_store reads AWS_ENDPOINT_URL as its S3 endpoint, and could otherwise pick it over the proxy's AWS_ENDPOINT. The cost: a script inside exec that sends other services to a custom endpoint loses it.
• Signed in: AWS_NO_SIGN_REQUEST and AWS_SKIP_SIGNATURE are removed, so GDAL and object_store sign even if they were exported for public buckets. The command gets AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN and AWS_CREDENTIAL_EXPIRATION. They're fetched once: cached ones are used only with at least 15 minutes left, otherwise new ones are minted (creds keeps its 60-second margin, since its callers ask again). A command still running when they expire must be run again; long jobs should use the profile.
• Signed out (no login session for the environment): the command runs unsigned.
  • stderr says Not signed in, so running unsigned: only public data can be read.
  • GDAL gets AWS_NO_SIGN_REQUEST=YES and object_store gets AWS_SKIP_SIGNATURE=true.
  • Every other AWS key is hidden from the command, because the proxy refuses keys it doesn't know even for public data (checked against the live proxy: a made-up key on s3://cholmes/admin-boundaries/ gets AccessDenied). AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_SECURITY_TOKEN, AWS_CREDENTIAL_EXPIRATION, AWS_ROLE_ARN, AWS_WEB_IDENTITY_TOKEN_FILE, AWS_ROLE_SESSION_NAME and the AWS_CONTAINER_* credential variables are removed; AWS_SHARED_CREDENTIALS_FILE and AWS_CONFIG_FILE point at /dev/null (NUL on Windows); AWS_EC2_METADATA_DISABLED=true. This also keeps a user's real AWS access key ID from being sent to the proxy.
  • The AWS CLI and s5cmd have no variable to skip signing. Without credentials they stop with Unable to locate credentials / NoCredentialProviders, so when the command is aws or s5cmd without --no-sign-request, stderr adds aws needs --no-sign-request to run unsigned: source-coop exec -- aws --no-sign-request s3 ls …, with the arguments quoted for the shell. exec doesn't add the flag itself: it doesn't rewrite the command it was given.
  • A product that isn't public is refused by the proxy (AccessDenied), and the command's exit code is passed on.
• Signed in, but the session can't be renewed (refresh token expired or revoked, sign-in server unreachable): an error, as for creds (Couldn't get new credentials (…). Run 'source-coop auth login'., exit 1), and the command isn't run. Someone who signed in means to act as themselves; silently running unsigned would turn that into a puzzling AccessDenied on their own data, or a write that fails. auth logout makes the unsigned run available again.
• creds signed out says Not signed in: run 'source-coop auth login'. (the same message the API commands give), followed by docs: say in --help where the data is and that public data needs no sign-in #101's Public data needs no credentials: … hint, and exits 4. A session that can't be renewed gets the same hint, through docs: say in --help where the data is and that public data needs no sign-in #101's mint_failed, for creds and exec alike. Before, it was Couldn't get new credentials (not logged in). …, exit 1. README Troubleshooting has the new message.
• No --json: the output is the command's. exec is listed with the other exceptions in the README.
• --help has examples, what the command gets and what happens signed out, and source-coop --help's summary for scripts points at exec for reading and writing data.

Demos

These run against the demo stub (docs/recorder/stub.py), not source.coop: record.py draws the stub's URLs as production's, and the records and keys are made up.

exec: the AWS CLI and s5cmd with your credentials and the proxy, no profile

exec signed out: unsigned, a public product listed, a restricted one refused

exec signed out: what GDAL, object_store and the AWS tools are told

exec: the command's exit code, a command that isn't there, and none at all

exec signed in, but the session can't be renewed: an error, not an unsigned run

Docs changes

• Reading data with AWS tools has a new subsection, Run a tool with exec: examples, what the command gets, the 15-minute credential rule, the signed-out behaviour and the error when a session can't be renewed, with two new demos (docs/demos/exec.svg, docs/demos/exec-signed-out.svg): https://cli.source.coop/pr-preview/pr-43/reading-data-with-aws-tools/#run-a-tool-with-exec
• Uploading data with AWS tools notes that exec writes too, signed in: https://cli.source.coop/pr-preview/pr-43/uploading-data-with-aws-tools/
• Scripts and AI agents lists exec among the commands without --json, and points at it for reading and writing data: https://cli.source.coop/pr-preview/pr-43/scripts-and-ai-agents/#json
• Configuration: SOURCE_ROLE_ARN applies to exec: https://cli.source.coop/pr-preview/pr-43/configuration/
• Troubleshooting: the Couldn't get new credentials entry names exec too, and a new entry covers Unable to locate credentials under exec: https://cli.source.coop/pr-preview/pr-43/troubleshooting/
• The intro and Quick start mention exec: https://cli.source.coop/pr-preview/pr-43/

Not in this PR

• Renewing credentials during a long command. They're fetched once; a command running past their expiry fails. Tools that support credential_process should use the profile, which renews them. Worth adding (for example a local credentials endpoint the child polls via AWS_CONTAINER_CREDENTIALS_FULL_URI) if people run multi-hour jobs through exec.
• Adding --no-sign-request to aws/s5cmd automatically. exec only suggests it, so the command runs exactly as typed. Worth revisiting if the suggestion proves to be noise.
• AWS_ENDPOINT_URL_STS. Not set: the command already holds credentials (or runs unsigned), so it never needs the proxy's STS, and a script's own STS calls keep going to AWS. service-account usage sets it because there the SDK exchanges a web identity token itself.
• Exit 4 for a session that can't be renewed. It exits 1, as creds always has; that message is shared with creds, and changing its classification belongs in its own PR.
• Exit 127/126 for a command that can't be found or run. It exits 1, "any other failure" in the CLI's exit-code table; matching shells and aws-vault exec would mean changing that table.
• A --duration of 15 minutes or less. Cached credentials then never meet exec's 15-minute minimum, so each run mints new ones, which still last less than 15 minutes. A design question about the minimum, not a local fix.
• Unsigned mode for boto3 / other SDK scripts. The AWS SDKs have no environment switch for unsigned requests; a script has to ask for it (botocore.UNSIGNED).

Follow-ups elsewhere

• source-data-proxy: a request signed with a key the proxy doesn't know is refused even for public data, so anyone with AWS keys in their environment can't read public products through the proxy without --no-sign-request. Treating an unknown key as anonymous (for reads of public products) would make this less surprising. exec works around it by hiding other keys.

Testing

• cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test and cargo test --features staging: 188 passed, 1 ignored, with docs: say in --help where the data is and that public data needs no sign-in #101 merged in from main. The exec tests are 7 in src/exec.rs (signed and unsigned environments, including the switches and endpoint removed; a local HTTP proxy; a region already set; the --no-sign-request hint, quoting both arguments and a program path with a space; no command) and 1 in src/main.rs (the notes printed). Whether anyone is signed in is now an Option from current_creds rather than a match on an error's text. The README's auth-logout demo is re-recorded for the new creds message, plus expires_within in src/cache.rs.
• uv run --with jsonschema python docs/recorder/check_stub.py: all routes match the staging spec. The docs site builds (docs/site/build.py).
• Manual, signed out, against the live production proxy, read-only, public data only: with a throwaway HOME and a made-up environment (a local read-only forwarder to https://data.source.coop, so no keychain item or real session was read), and fake AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/AWS_PROFILE in the environment: aws --no-sign-request s3 ls, s5cmd --no-sign-request ls, GDAL ogrinfo /vsis3/… and obstore S3Store(...).list() all listed s3://cholmes/admin-boundaries/; no request reached the proxy signed; aws s3 ls without the flag printed the hint and failed with Unable to locate credentials; sh -c 'exit 7' exited 7.
• Not tried: the signed-in path against a real or staging proxy (it needs a real login, so the keychain); the signed-in path was run against the stub only (demos above). Windows was not tried.

🤖 Generated with Claude Code

[claude]

Claude finished @alukach's task in 24s —— View job

---

✅ No blocking issues — safe to merge.

I read the diff in full (src/exec.rs, src/main.rs, src/cache.rs, src/api.rs, src/sts.rs) and did not build it or run the tests.

• Credentials: current_creds re-checks the cache under the lock before minting, the same as the old run_creds did. A session that can't be renewed is an error, not None, so a signed-in user is never silently run unsigned.
• Unsigned mode: it hides the other AWS keys and points the credentials files at /dev/null, so a real AWS key ID can't reach the proxy.
• Process handling: on Unix, exec replaces the process after the lock guard has already dropped, so no lock is held while the command runs. Exit codes and signals pass straight through.

Minor points, none blocking:

• Signed-in, leftover credential variables: AWS_ROLE_ARN, AWS_WEB_IDENTITY_TOKEN_FILE and AWS_CONTAINER_* are only removed when unsigned. Static AWS_ACCESS_KEY_ID wins in the SDK chain, so it works today. Moving OTHER_CREDENTIALS minus the four keys into the always-removed list would make signed-in runs match the documented precedence for every tool.
• mint_failed: the "Run 'source-coop auth login'" text now appears for exec failures where a refresh token was revoked. That's correct, but the sign-in server being down gives the same advice. This already applied to creds.
• allow_hyphen_values on the command: a mistyped flag before the command, such as exec --role-arn=x aws … written as exec aws --role-arn x, is passed to the child instead of being rejected. That's inherent to the trailing-args design, and the docs already say everything after -- is the command.

Simplify (ponytail)

• src/main.rs, CREDS_MIN_LIFETIME / EXEC_MIN_LIFETIME: both are used once, so inlining Duration::seconds(60) and Duration::minutes(15) would work. The names do document the intent, so this is optional.
• src/main.rs, public_data + mint_failed: two helpers for one format string. Inlining public_data would work, but it is also used by run_creds, so keep it.
• src/exec.rs, Env { set, remove }: it exists so env() can be unit tested, which is reasonable. Nothing to cut.

---

💰 Estimated review cost: $0.22 · 0m24s · 6 turns

[github-actions]

Docs preview: https://cli.source.coop/pr-preview/pr-43/

Built from b04e957. It goes live a minute or two after this run, once GitHub Pages deploys it, and is removed when the PR closes.