diff --git a/docs/credit-costs/index.html b/docs/credit-costs/index.html index 34032ae6..f8b6393f 100644 --- a/docs/credit-costs/index.html +++ b/docs/credit-costs/index.html @@ -321,11 +321,11 @@

Resources

search -1 credit +2 credits search + scrape -1 credit + 1 per scraped result +2 credits + 1 per scraped result monitor create / list / get @@ -349,7 +349,7 @@

Resources

Top-up credits

Purchased top-up credits never expire and are not tied to your current billing cycle. Cancelling your subscription zeroes only the plan's included/monthly credit allowance; any purchased top-up balance survives cancellation and stays spendable.

Free tier

-

The FREE plan grants 1000 credits once, for the lifetime of the account, not monthly, and it never resets. No card is required to get them. Once they are spent, a FREE account either upgrades to a paid plan or stops.

+

The FREE plan grants 500 credits once, for the lifetime of the account, not monthly, and it never resets, plus up to 500 more from onboarding tasks. No card is required to get them. Once they are spent, a FREE account either upgrades to a paid plan or stops.

Why crawl billing looks different

The crawl start reserves the job. Subsequent polls charge only for newly materialized pages, not for the total accumulated page count each time.

That prevents the same already-seen pages from being charged again and again just because you are checking progress.

@@ -383,15 +383,15 @@

Simple Examples

Search for "AI tools" with 5 results -1 credit +2 credits Search + scrape 3 results -1 + 3 = 4 credits +2 + 3 = 5 credits Search + scrape, 1 scrape fails -1 + 2 = 3 credits (failed scrape refunded) +2 + 2 = 4 credits (failed scrape refunded)

What Usually Does Not Consume Permanent Credits

diff --git a/docs/docker/index.html b/docs/docker/index.html index 4da5cf8e..35451fbd 100644 --- a/docs/docker/index.html +++ b/docs/docker/index.html @@ -289,7 +289,8 @@

Pre-built Image

docker run -p 3000:3000 ghcr.io/us/crw:0.16.0

Available tags: latest, 0.16 (tracks the current minor), 0.16.0 (pinned). Use a pinned tag in production — latest rolls forward on every release.

-

Docker Compose

+
500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key

Docker Compose

git clone https://github.com/us/crw.git
 cd crw
 docker compose up
diff --git a/docs/docs/credit-costs.md b/docs/docs/credit-costs.md index f55f3563..00c00c6d 100644 --- a/docs/docs/credit-costs.md +++ b/docs/docs/credit-costs.md @@ -15,8 +15,8 @@ Cloud only (fastcrw.com) -- self-hosted instances do not have credit-based billi | `map` | 1 credit | | `crawl` start | 1 credit | | `crawl` polling | New pages discovered since the previous poll | -| `search` | 1 credit | -| `search` + scrape | 1 credit + 1 per scraped result | +| `search` | 2 credits | +| `search` + scrape | 2 credits + 1 per scraped result | | `monitor` create / list / get | 0 credits | | `monitor` check (per run) | 1 credit per scraped page, plus 1 more credit per page judged changed | | `browse` | Not a billed cloud endpoint. It is a local CLI/companion capability, free either way | @@ -43,7 +43,7 @@ Purchased top-up credits never expire and are not tied to your current billing c ## Free tier -The FREE plan grants 1000 credits **once, for the lifetime of the account**, not monthly, and it never resets. No card is required to get them. Once they are spent, a FREE account either upgrades to a paid plan or stops. +The FREE plan grants 500 credits **once, for the lifetime of the account**, not monthly, and it never resets, plus up to 500 more from onboarding tasks. No card is required to get them. Once they are spent, a FREE account either upgrades to a paid plan or stops. ## Why crawl billing looks different @@ -60,9 +60,9 @@ That prevents the same already-seen pages from being charged again and again jus | Start one crawl job | 1 credit | | Poll a crawl and receive 7 new pages | 7 additional credits | | Poll again with no new pages | No new page credits | -| Search for "AI tools" with 5 results | 1 credit | -| Search + scrape 3 results | 1 + 3 = 4 credits | -| Search + scrape, 1 scrape fails | 1 + 2 = 3 credits (failed scrape refunded) | +| Search for "AI tools" with 5 results | 2 credits | +| Search + scrape 3 results | 2 + 3 = 5 credits | +| Search + scrape, 1 scrape fails | 2 + 2 = 4 credits (failed scrape refunded) | ## What Usually Does Not Consume Permanent Credits diff --git a/docs/docs/docker.md b/docs/docs/docker.md index b5612ea6..4f61cf2e 100644 --- a/docs/docs/docker.md +++ b/docs/docs/docker.md @@ -12,6 +12,11 @@ docker run -p 3000:3000 ghcr.io/us/crw:0.16.0 Available tags: `latest`, `0.16` (tracks the current minor), `0.16.0` (pinned). Use a pinned tag in production — `latest` rolls forward on every release. +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ## Docker Compose ```bash diff --git a/docs/docs/glossary.md b/docs/docs/glossary.md index 2401aaac..32bdb097 100644 --- a/docs/docs/glossary.md +++ b/docs/docs/glossary.md @@ -30,7 +30,7 @@ Chrome DevTools Protocol is the WebSocket-based protocol that lets external prog ## credit -A credit is the billing unit for the hosted cloud at `fastcrw.com`, worth $0.001. Every scrape costs 1 credit regardless of renderer (HTTP, lightpanda, or Chrome); there is no renderer or proxy surcharge. LLM-backed work (`POST /v1/extract`, or `formats: ["json"]` / `summary` on scrape) costs that 1-credit base render plus the actual LLM cost of the call, dynamically metered: there is no flat credit number for it. See the [Credit Costs](/docs/credit-costs) table for current billing. Map and search start-calls each cost 1 credit, and crawl jobs charge 1 credit at start plus one additional credit per page as pages are discovered during polling. The FREE plan grants 1000 credits once, for the lifetime of the account, not monthly, and they never reset. Purchased top-up credits never expire and survive plan cancellation. Self-hosted deployments have no billing layer and are unaffected by credit costs. Check your balance with `GET /api/v1/account/balance` on the SaaS control-plane. See [Credit Costs](/docs/credit-costs) for the full table. +A credit is the billing unit for the hosted cloud at `fastcrw.com`, worth $0.001. Every scrape costs 1 credit regardless of renderer (HTTP, lightpanda, or Chrome); there is no renderer or proxy surcharge. LLM-backed work (`POST /v1/extract`, or `formats: ["json"]` / `summary` on scrape) costs that 1-credit base render plus the actual LLM cost of the call, dynamically metered: there is no flat credit number for it. See the [Credit Costs](/docs/credit-costs) table for current billing. A map call costs 1 credit, a search call costs 2 credits, and crawl jobs charge 1 credit at start plus one additional credit per page as pages are discovered during polling. The FREE plan grants 500 credits once, for the lifetime of the account, not monthly, and they never reset, plus up to 500 more from onboarding tasks. Purchased top-up credits never expire and survive plan cancellation. Self-hosted deployments have no billing layer and are unaffected by credit costs. Check your balance with `GET /api/v1/account/balance` on the SaaS control-plane. See [Credit Costs](/docs/credit-costs) for the full table. --- diff --git a/docs/docs/installation.md b/docs/docs/installation.md index 7164387c..d59be73a 100644 --- a/docs/docs/installation.md +++ b/docs/docs/installation.md @@ -2,7 +2,9 @@ ## Cloud (no installation needed) -Sign up at [fastcrw.com](https://fastcrw.com) and start using the API immediately. +Skip installation and get a free API key for fastCRW Cloud, or install the self-hosted engine as a single binary, Docker container, or Homebrew formula. + +Sign up at [fastcrw.com/register](https://fastcrw.com/register?ref=docs-installation-inline) and start using the API immediately. Native `/v1` endpoints for new CRW integrations, with `/firecrawl/v2` compatibility available for Firecrawl migrations. ```bash @@ -12,6 +14,11 @@ curl -X POST https://api.fastcrw.com/v1/scrape \ -d '{"url": "https://example.com"}' ``` +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ## One-Line Install (Recommended) The install script auto-detects your OS and architecture, downloads the latest binary, and installs it: diff --git a/docs/docs/introduction.md b/docs/docs/introduction.md index 0ea10d45..d883c3a0 100644 --- a/docs/docs/introduction.md +++ b/docs/docs/introduction.md @@ -3,7 +3,7 @@
Get Started

CRW Docs

-

Turn websites into usable data with one API. Start with a single scrape request, then move into search, map, crawl, extract, or MCP only when your workflow actually needs them. Interactive browser automation is handled by the companion crw-browse service.

+

Open-source web scraping API for agents and RAG. Scrape, crawl, search, map, and extract data from any site with one Rust binary, self-hosted or on fastCRW Cloud.

Fastest first win: one URL, one markdown response
Works for: agents, ETL, RAG, structured extraction
@@ -15,6 +15,27 @@
+## Setting up an AI coding agent? + +Paste this into Claude Code, Cursor, Codex, Windsurf, or any other coding agent's chat: + +:::note +Install fastCRW for web access: run npx crw-mcp@latest install to add the MCP server and skill, then use the crw_scrape, crw_search, crw_map, and crw_crawl tools (or the native /v1 REST API) to fetch, search, and crawl the web with the CRW_API_KEY in my environment. +::: + +Full reference written for autonomous agents: [Agent Onboarding Guide](/agent-onboarding). + +## Pick your endpoint + +:::cards +::card{icon="code" title="Scrape" href="#scraping" description="One known URL in, clean markdown or JSON out."} +::card{icon="search" title="Search" href="#search" description="No URL yet? Search the web and get ranked results with content."} +::card{icon="map" title="Map" href="#map" description="Discover every URL under a domain before you scrape or crawl."} +::card{icon="layers" title="Crawl" href="#crawling" description="Scrape every page under a URL in one background job."} +::card{icon="key" title="Extract" href="#extract" description="Pull structured JSON fields from a page with an LLM."} +::card{icon="plug" title="MCP" href="#mcp" description="Give Claude, Cursor, Codex, and other agents live web access."} +::: + > **New to CRW? Use `/v1`.** The `/v1` routes are the native fastCRW API for new integrations. Use `/firecrawl/v2` when migrating existing Firecrawl v2 SDK code or when you need compatibility-only routes such as batch scrape or PDF parse.
@@ -23,6 +44,8 @@
If this request works, you already understand the core CRW model: known URL in, clean content out. Everything else in the docs builds on that.
+**Try it without a key:** run this exact request in the playground (rate-limited preview, no signup). + ## Does this work for me? | I want to… | Use | diff --git a/docs/docs/js-rendering.md b/docs/docs/js-rendering.md index 73895ab0..b9a7c872 100644 --- a/docs/docs/js-rendering.md +++ b/docs/docs/js-rendering.md @@ -40,6 +40,11 @@ Override the rendering mode per request using `renderJs`: | `true` | Force CDP rendering | | `false` | HTTP only | +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ## Global default To force JS rendering for every request that doesn't specify `renderJs` explicitly, set `render_js_default` in your config: diff --git a/docs/docs/map.md b/docs/docs/map.md index eea516b3..0a1341ab 100644 --- a/docs/docs/map.md +++ b/docs/docs/map.md @@ -23,6 +23,8 @@ +**Try it without a key:** run this exact request in the playground (rate-limited preview, no signup). + ## Mapping a site with CRW ### /v1/map diff --git a/docs/docs/mcp-clients.md b/docs/docs/mcp-clients.md index a303716e..379bfec3 100644 --- a/docs/docs/mcp-clients.md +++ b/docs/docs/mcp-clients.md @@ -67,6 +67,11 @@ If you only remember one rule, remember this one: local embedded mode is the eas claude mcp add crw -- npx -y crw-mcp ``` +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ### fastcrw.com cloud ```bash diff --git a/docs/docs/mcp.md b/docs/docs/mcp.md index e37045bc..0d8d2dec 100644 --- a/docs/docs/mcp.md +++ b/docs/docs/mcp.md @@ -64,6 +64,11 @@ That's it. The agent starts `crw-mcp`, which contains the full scraping engine. If you want host-by-host config files instead of one-liners, jump to [MCP Client Setup](#mcp-clients). +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ### With CDP rendering (LightPanda/Chrome) If you have a CDP-compatible browser, pass it via env vars: diff --git a/docs/docs/quick-start.md b/docs/docs/quick-start.md index 94e63457..b98b278f 100644 --- a/docs/docs/quick-start.md +++ b/docs/docs/quick-start.md @@ -5,7 +5,7 @@
Goal: first success in under 3 minutes
Base URL: https://api.fastcrw.com
-
Free tier: 1000 credits, no card required
+
Free tier: 500 credits to start, up to 500 more from onboarding tasks, no card required
Get API key @@ -22,13 +22,13 @@ - **Terminal** — macOS Terminal, Linux shell, or Windows [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) - **`curl`** — ships with macOS 10.15+ and most Linux distros; Windows users can use WSL or [download curl](https://curl.se/windows/) -- **A free account** at [fastcrw.com/register](https://fastcrw.com/register?ref=docs-quick-start-inline) — 1000 credits, one-time, no card required +- **A free account** at [fastcrw.com/register](https://fastcrw.com/register?ref=docs-quick-start-inline): 500 credits to start, and up to 500 more from onboarding tasks, no card required - **Node.js 18+** — only for the MCP path (optional, described at the bottom of this page) ::: ## Get a key -Register at [fastcrw.com/register](https://fastcrw.com/register?ref=docs-quick-start-inline). Once you confirm your email, your API key appears on the dashboard. Your account starts with **1000 free credits** — one credit equals one basic scrape request, so you have plenty to explore. +Register at [fastcrw.com/register](https://fastcrw.com/register?ref=docs-quick-start-inline). Once you confirm your email, your API key appears on the dashboard. Your account starts with **500 credits, and up to 500 more from onboarding tasks**. one credit equals one basic scrape request, so you have plenty to explore. Copy the key and keep it somewhere safe. You will paste it into the `Authorization` header below. diff --git a/docs/docs/rate-limits.md b/docs/docs/rate-limits.md index 17faacd5..85e45559 100644 --- a/docs/docs/rate-limits.md +++ b/docs/docs/rate-limits.md @@ -69,7 +69,7 @@ You already have as many requests in flight as your plan allows. The response in ### Cause 3: Credits exhausted -Once your available balance reaches zero, the API returns `429`. This covers a FREE account that has spent its lifetime 1000 credits, and a paid account that is out of credits without an active auto-recharge attempt in progress. The response includes `X-FASTCRW-Credits-Available`, `X-FASTCRW-Included-Remaining`, `X-FASTCRW-Purchased-Remaining`, and `X-FASTCRW-Upgrade-Url` headers. There is no reset header: a FREE account's 1000 credits are a one-time lifetime grant, not a monthly allowance, so there is nothing to reset. +Once your available balance reaches zero, the API returns `429`. This covers a FREE account that has spent its lifetime 500 credits (plus up to 500 more from onboarding tasks), and a paid account that is out of credits without an active auto-recharge attempt in progress. The response includes `X-FASTCRW-Credits-Available`, `X-FASTCRW-Included-Remaining`, `X-FASTCRW-Purchased-Remaining`, and `X-FASTCRW-Upgrade-Url` headers. There is no reset header: a FREE account's credits are a one-time lifetime grant, not a monthly allowance, so there is nothing to reset. **Action:** do NOT retry. Retrying burns no credits (the request is rejected before processing) but creates noise in your logs and may hide the real cause. Alert or pause your pipeline and top up your balance or upgrade your plan. diff --git a/docs/docs/recipe-mcp-5min.md b/docs/docs/recipe-mcp-5min.md index 852b1b0f..48039fba 100644 --- a/docs/docs/recipe-mcp-5min.md +++ b/docs/docs/recipe-mcp-5min.md @@ -24,7 +24,7 @@ Claude Code writes this into your project `.claude/mcp.json` automatically. You ### Option B: Cloud mode (all 9 tools, including `crw_search`) -Get a free API key at [fastcrw.com](https://fastcrw.com) — 1000 one-time lifetime credits, no monthly reset. +Get a free API key at [fastcrw.com](https://fastcrw.com): 500 credits to start, and up to 500 more from onboarding tasks, no monthly reset. ```bash claude mcp add crw \ diff --git a/docs/docs/scraping.md b/docs/docs/scraping.md index db39395c..3e6660a5 100644 --- a/docs/docs/scraping.md +++ b/docs/docs/scraping.md @@ -23,6 +23,8 @@
+**Try it without a key:** run this exact request in the playground (rate-limited preview, no signup). + ## Scraping a URL with CRW ### /v1/scrape @@ -104,6 +106,11 @@ curl -X POST https://api.fastcrw.com/v1/scrape \ ``` ::: +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ### Response ```json diff --git a/docs/docs/sdk-reference.md b/docs/docs/sdk-reference.md index 18e007ca..8b8031de 100644 --- a/docs/docs/sdk-reference.md +++ b/docs/docs/sdk-reference.md @@ -52,7 +52,7 @@ const crw = new CrwClient(); const crw = new CrwClient({ apiKey: "crw_live_..." }); ``` -Sign up for **1000 free credits** (no payment card, no monthly reset) at +Sign up for **500 free credits, and up to 500 more from onboarding tasks** (no payment card, no monthly reset) at and then set `CRW_API_KEY` in your environment. ### Self-hosted HTTP server diff --git a/docs/docs/search.md b/docs/docs/search.md index 455c2a94..02c53214 100644 --- a/docs/docs/search.md +++ b/docs/docs/search.md @@ -24,6 +24,8 @@ +**Try it without a key:** run this exact request in the playground (rate-limited preview, no signup). + :::note **Self-hosted users**: `docker compose up` boots the search sidecar automatically (reachable inside the Compose network as `searxng:8080`). `/v1/search` is live on `http://localhost:3000` with no extra setup. To point at a search backend you already run instead, set `CRW_SEARCH__SEARCH_BACKEND_URL=http://your-host:8080` and remove the `searxng` service from your compose file. To disable search entirely, set `[search].enabled = false` — the route returns a clear `search_disabled` error (HTTP 503). See the [Docker → Search backend](/docker) section for the full setup, the `SEARXNG_BASE_URL` vs `search_backend_url` distinction, and cold-start timing. ::: @@ -116,6 +118,11 @@ curl -X POST https://api.fastcrw.com/v1/search \ ``` ::: +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ### Response ```json diff --git a/docs/docs/self-hosting.md b/docs/docs/self-hosting.md index 4538da81..949ba278 100644 --- a/docs/docs/self-hosting.md +++ b/docs/docs/self-hosting.md @@ -18,6 +18,11 @@ curl -X POST http://localhost:3000/v1/scrape \ -d '{"url":"https://example.com","formats":["markdown"]}' ``` +:::note +500 credits to start, up to 500 more from onboarding tasks, no card. +Get your API key +::: + ## What You Get - the same core self-hosted routes: `scrape`, `crawl`, `map`, `mcp`, `health` diff --git a/docs/glossary/index.html b/docs/glossary/index.html index 15c92fad..d3534600 100644 --- a/docs/glossary/index.html +++ b/docs/glossary/index.html @@ -298,7 +298,7 @@

chunkStrategy

chunkStrategy is an optional request field on POST /v1/scrape that tells the engine to split the page's markdown into smaller pieces server-side before returning it. You supply an object with a type ("topic" to split on headings, "sentence" to split on punctuation, or "regex" for a custom pattern), an optional maxChars cap per chunk, and an optional dedupe flag to drop near-duplicate chunks. The engine returns the chunks as a chunks array alongside the normal markdown field — no text-splitting library needed on the client side. Note that chunkStrategy only works on POST /v1/scrape; it is not forwarded by batch or crawl jobs. See the RAG recipe for a complete example.


credit

-

A credit is the billing unit for the hosted cloud at fastcrw.com, worth $0.001. Every scrape costs 1 credit regardless of renderer (HTTP, lightpanda, or Chrome); there is no renderer or proxy surcharge. LLM-backed work (POST /v1/extract, or formats: ["json"] / summary on scrape) costs that 1-credit base render plus the actual LLM cost of the call, dynamically metered: there is no flat credit number for it. See the Credit Costs table for current billing. Map and search start-calls each cost 1 credit, and crawl jobs charge 1 credit at start plus one additional credit per page as pages are discovered during polling. The FREE plan grants 1000 credits once, for the lifetime of the account, not monthly, and they never reset. Purchased top-up credits never expire and survive plan cancellation. Self-hosted deployments have no billing layer and are unaffected by credit costs. Check your balance with GET /api/v1/account/balance on the SaaS control-plane. See Credit Costs for the full table.

+

A credit is the billing unit for the hosted cloud at fastcrw.com, worth $0.001. Every scrape costs 1 credit regardless of renderer (HTTP, lightpanda, or Chrome); there is no renderer or proxy surcharge. LLM-backed work (POST /v1/extract, or formats: ["json"] / summary on scrape) costs that 1-credit base render plus the actual LLM cost of the call, dynamically metered: there is no flat credit number for it. See the Credit Costs table for current billing. A map call costs 1 credit, a search call costs 2 credits, and crawl jobs charge 1 credit at start plus one additional credit per page as pages are discovered during polling. The FREE plan grants 500 credits once, for the lifetime of the account, not monthly, and they never reset, plus up to 500 more from onboarding tasks. Purchased top-up credits never expire and survive plan cancellation. Self-hosted deployments have no billing layer and are unaffected by credit costs. Check your balance with GET /api/v1/account/balance on the SaaS control-plane. See Credit Costs for the full table.


Firecrawl-compatible

"Firecrawl-compatible" means fastCRW exposes a /firecrawl/v2 compatibility layer for existing Firecrawl v2 SDK integrations. It is intentionally close enough for migration work, but it is not the recommended API for new fastCRW builds. New projects should start with /v1; migration projects should validate the documented differences before switching production traffic. See Migrate from Firecrawl for the exact list of changes and Compatibility for the behavior matrix.

diff --git a/docs/installation/index.html b/docs/installation/index.html index 21ea5a09..191297f9 100644 --- a/docs/installation/index.html +++ b/docs/installation/index.html @@ -3,8 +3,8 @@ - Installation — CRW Docs - + Install: CLI, Docker, or Self-Host — CRW Docs + @@ -14,14 +14,14 @@ - - + + - - + +