Skip to content

docs: say in --help where the data is and that public data needs no sign-in - #101

Merged
alukach merged 3 commits into
mainfrom
docs/help-reading-data
Nov 9, 2026
Merged

alukach merged 3 commits into
mainfrom
docs/help-reading-data

Conversation

@alukach

@alukach alukach commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

An agent (or person) that learns the CLI from --help can now find out how to read data: products are at s3://ACCOUNT/PRODUCT/ on the data proxy, public data needs no sign-in, and each tool's settings (AWS CLI, DuckDB, GDAL, Python) are on docs.source.coop/data-proxy. --help keeps a two-line summary and links there rather than repeating it; the CLI documents only its own part, the credentials. Before, nothing in --help said where the data was, --proxy-url read as if it only picked a login session, and signed-out creds sent people to a browser login they didn't need for public data.

source-coop --help         # "For scripts and agents" gains a Data entry
source-coop creds --help   # the AWS profile, and the docs link for reading data

# public data, no sign-in or credentials
aws s3 ls s3://ACCOUNT/PRODUCT/ --endpoint-url https://data.source.coop --no-sign-request

How it behaves

  • source-coop --help: the "For scripts and agents" summary has a two-line Data entry: s3://ACCOUNT/PRODUCT/ on the proxy, public data needs no sign-in, creds --help for credentials, and https://docs.source.coop/data-proxy for each tool's settings.
  • source-coop creds --help ends with examples (source-coop creds, --format env), the ~/.aws/config profile (credential_process = source-coop creds, endpoint_url) with aws s3 ls … --profile source-coop, and a line linking https://docs.source.coop/data-proxy for public data and other tools.
  • The URLs in both are the build's default proxy: https://data.source.coop, or https://data.staging.source.coop in staging builds (substituted like {api} already is).
  • --proxy-url's help on creds and auth now says it's the S3 endpoint of the data for AWS tools, as well as what selects the environment. On the API commands (product, member, …), where it only picks the login session, it says just that.
  • Signed-out creds adds to its error: Public data needs no credentials: read it with \aws s3 ... --endpoint-url URL --no-sign-request`.` It follows every failure to get credentials (signed out, a session that expired or was revoked, STS refusing), since public reads need none in any of those cases. Exit code is unchanged (1).

Demos

The demos run against docs/recorder/stub.py, not a real environment. The recorder draws the stub's URLs as production's (data.source.coop), so the records in them (demo-org) are made up, not real ones.

source-coop creds --help: the AWS profile, and a link to reading data

source-coop creds --help: the AWS profile, and a link to reading data

source-coop --help: the Data entry for scripts and agents

source-coop --help: the Data entry for scripts and agents

Reading public data without signing in, and creds signed out

Reading public data without signing in, and creds signed out

Docs changes

  • Reading data with AWS tools: an intro saying where the data is, that public data needs no sign-in, and linking Access Data for reading it with each tool. The rest of the section is the CLI's part: the credential_process profile, formats and roles.
  • Scripts and AI agents: a short "Read data" subsection, matching the new Data entry in --help's summary, with the same link.
  • Configuration: --proxy-url / SOURCE_PROXY_URL described as the S3 endpoint for AWS tools as well as the environment.
  • Troubleshooting: the Couldn't get new credentials entry mentions the public-data note and links Access Data.

Not in this PR

  • Per-tool recipes (anonymous reads, DuckDB, GDAL, Python) live on docs.source.coop, not here: docs: read data through the proxy with DuckDB, GDAL and Python docs.source.coop#49 adds DuckDB, GDAL and Python to the Access Data page. Until it merges, that page covers the AWS CLI only.

  • eval $(source-coop creds --format env) still sets no endpoint, so a bare aws s3 ls after it goes to AWS. New --format values and an exec command are in separate PRs, so creds --help doesn't document them here.

  • The signed-out exit code stays 1; changing it to 4 (not signed in) would be a separate, behavioural change.

Testing

  • cargo fmt, cargo clippy --all-targets -- -D warnings, cargo test (179 passed), cargo test --features staging (179 passed). Docs site built (--with pygments is now needed too). One test added (mint_failed_mentions_public_data), and the README's auth-logout demo re-recorded for the new message. Otherwise the new help goes through the existing checks (help_examples_parse parses the two source-coop creds examples, help_headings_are_styled_like_clap_s, every_command_and_argument_has_help).
  • Built the docs site locally and checked the new anchors and links.
  • The anonymous aws s3 ls --no-sign-request was confirmed against the live production proxy while researching this; the demo runs it against the stub. The staging help text wasn't checked by eye beyond the staging test run.

🤖 Generated with Claude Code

…ign-in

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @alukach's task in 9s —— View job


✅ No blocking issues — safe to merge.

I reviewed the diff in src/main.rs and README.md and found no correctness or security problems.

  • The {proxy} substitution in FOR_SCRIPTS and CREDS_HELP mirrors the existing {api} handling. mint_failed is a small pure function with a test.
  • The README, --help, --proxy-url help and Configuration table agree with each other.
  • The PR description is out of date. "How it behaves" says the public-data note appears "only when the reason is not logged in". After commit 92f0984, mint_failed (src/main.rs:1022) adds it to every mint failure (expired session, STS refusal and so on). The code, its doc comment, the test and the README ("The message also says…") all describe the newer behaviour, so only the description needs fixing.
  • The message tells people to run aws s3 ... --endpoint-url URL --no-sign-request. The ... is literal placeholder text. That is fine for a hint.

Simplify (ponytail)

  • src/main.rs CREDS_HELP / FOR_SCRIPTS: the two .replace("{proxy}", …) chains could share one small helper. With two call sites this is not worth doing yet.

💰 Estimated review cost: $0.12 · 0m09s · 4 turns

github-actions Bot added a commit that referenced this pull request Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Docs preview: https://cli.source.coop/pr-preview/pr-101/

Built from 92f0984. It goes live a minute or two after this run, once GitHub Pages deploys it, and is removed when the PR closes.

How to read data through the proxy (anonymously, and from DuckDB, GDAL
and Python) is a fact about source.coop, documented on
docs.source.coop/data-proxy. --help keeps a two-line summary and the
link; creds --help keeps the CLI's own part, the AWS profile. The
README's Public data subsection becomes a link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions Bot added a commit that referenced this pull request Nov 9, 2026
…er command

Signed-out creds said public data needs no credentials, but an expired or
revoked session didn't, though public reads need none then either. The
hint now follows every failure to get credentials, built in one tested
function rather than by matching mint's error text.

On API commands (product, member, ...) --proxy-url only picks the login
session, so its help says that instead of calling it the S3 endpoint.

Re-record the README's auth-logout demo, which showed the old message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions Bot added a commit that referenced this pull request Nov 9, 2026
@alukach
alukach merged commit b594138 into main Nov 9, 2026
12 checks passed
@alukach
alukach deleted the docs/help-reading-data branch November 9, 2026 18:18
github-actions Bot added a commit that referenced this pull request Nov 9, 2026
alukach added a commit that referenced this pull request Nov 9, 2026
Keep exec's current_creds for creds, with #101's public-data hint on both
of its failures: signed out ("Not signed in: …", exit 4) and a session
that can't be renewed (mint_failed). The README's Read data subsection
mentions exec, and the auth-logout demo is re-recorded with the hint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alukach added a commit that referenced this pull request Nov 9, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alukach added a commit that referenced this pull request Nov 9, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant