v0.3.0

v0.3.0

Scraper Studio gets a major workflow upgrade: multi-URL runs, a stable
machine-readable output envelope, automatic backoff on the concurrent-job
cap, real CSV/HTML/Markdown file output, clearer error hints, and an
Examples: section in every command's --help.

All changes are backward compatible — existing bdata scraper run <id> <url>
and scraper create invocations behave exactly as before.

Features

scraper run — multi-URL input (#8)

Run a scraper against many URLs in a single API call. The CLI was the only
client without this; it now mirrors the reference SDKs' batch path.

• --urls "u1,u2,u3" — comma-separated list.
• --input-file <path> — one URL per line (# comments and blanks skipped),
  or a JSON array of strings, or a JSON array of {"url": "..."} objects.

A list of URLs becomes one POST /dca/trigger, one snapshot, one merged
result array. The positional <url> is now optional; exactly one input
source may be given. --sync is rejected for multi-URL (the sync endpoint is
single-URL only).

bdata scraper run c_xxx --urls "https://a.test/1,https://a.test/2" -o out.json
bdata scraper run c_xxx --input-file urls.txt -o out.json

scraper create — stable output envelope (#6)

-o / --json now writes a consistent envelope on every termination path
(success and failure), so the documented jq -r '.collector_id' recipe always
finds an id — even when a build half-fails:

{
  "collector_id": "c_...",
  "name": "...",
  "status": "done | failed | ai_trigger_failed | poll_failed",
  "completed_steps": ["..."],
  "view_url": "https://brightdata.com/cp/scrapers/c_...",
  "created_at": "...",
  "error": "..."
}

Failure paths that previously wrote nothing now write the envelope (with a
recoverable collector_id). --legacy-output keeps the old bare-progress
shape for one minor version while you migrate.

scraper create — auto-backoff on the AI-Flow 429 cap (#7)

AI generation is capped at a few concurrent jobs per account. Hitting the cap
used to fail within seconds, leaving half-built collectors behind. The CLI now
waits with exponential backoff + jitter and retries, printing status so you
know it isn't hung:

Hit AI-Flow concurrent-job cap (429). Waiting 32s before retry 1/4...

• --max-retries <n> — override the retry count (default 4).
• --no-retry — fail fast on 429 instead of waiting.

On terminal failure, a recovery note points at the half-built collector's
dashboard URL.

Output — real CSV / HTML / Markdown file formats (#9)

-o file.csv now writes actual CSV, file.md writes a Markdown table, and
file.html writes an HTML table — chosen by file extension.

bdata scraper run c_xxx https://a.test -o products.csv   # RFC-4180 CSV

.xlsx / .xls are rejected up front with a helpful message instead of
silently producing a broken file.

--help — Examples in every command (#10)

scraper create, scraper run, discover, search, pipelines, and
scrape each gain an Examples: section in --help, using real public
domains. No more alt-tabbing to the README to find a working invocation.

bdata scraper create --help   # ends with copy-paste-ready examples

Fixes

Accurate error hints for Scraper Studio (#5)

A 403 from a stub collector used to map to a misleading "check your zone
permissions" hint, sending users down the wrong path. Scraper-API errors now
get accurate, actionable hints (e.g. "AI generation has not completed — re-run
scraper create", "you hit the AI-Flow concurrent-job cap"). Other commands
(scrape, search, discover, pipelines, browser) are unchanged — the
scraper vocabulary stays scoped to the scraper command.

Bug fix

--no-retry on scraper create was silently ignored due to a Commander flag
mapping (opts.retry vs opts.noRetry); it now correctly disables retries.

Upgrade notes

• No action required — fully backward compatible.
• If you parse the scraper create -o file, note it is now an envelope, not
  the bare AI-progress payload. Use --legacy-output to keep the old shape
  during migration (removed next minor).

Full changelog: v0.2.0...v0.3.0 (PRs #5–#10)