Skip to content

[SCR-383] Auto-Mode (mode=auto) support#3

Open
kostas-jakeliunas-sb wants to merge 1 commit into
masterfrom
SCR-383/auto-mode-support
Open

[SCR-383] Auto-Mode (mode=auto) support#3
kostas-jakeliunas-sb wants to merge 1 commit into
masterfrom
SCR-383/auto-mode-support

Conversation

@kostas-jakeliunas-sb

@kostas-jakeliunas-sb kostas-jakeliunas-sb commented Jun 30, 2026

Copy link
Copy Markdown

Merge order: part of the [SCR-383] Auto-Mode client rollout — merge once the server PR ScrapingBee/scrapingbee-API#1242 is merged + the flag is enabled.

tl;dr

Docs-only change so a LangChain agent can choose ScrapingBee Auto-Mode (mode=auto). The tools already spread arbitrary params into the request, so mode/max_cost work today — the LLM just won't select them until the tool docstring lists them. No request-code or architecture change; no validation added.

Summary

  • langchain_scrapingbee/tools.py — added mode and max_cost to ScrapeUrlTool's LLM-facing docs so an agent will pick Auto-Mode:
    • new mode + max_cost entries in the SUPPORTED PARAMS list (the existing UNSUPPORTED PARAMS: anything not listed clause now permits them);
    • a short prose hint at the top of the tool description ("when unsure which config a site needs, prefer mode='auto'…");
    • a {"mode": "auto", "max_cost": 25} example in the ScrapeUrlInput.params field description.
    • Accurate contract documented: mode='auto' is GET-only, cheapest→expensive, charges only the winning config (0 on full failure), reports credits via the Spb-auto-cost response header, and must not be combined with render_js/premium_proxy/stealth_proxy/transparent_status_code; max_cost (int ≥ 1) caps spend and requires mode='auto'.
  • README.md — added an Auto-Mode usage example (Test Case 1b) mirroring the existing examples, passing params={"mode":"auto","max_cost":25} to ScrapeUrlTool.
  • pyproject.toml / release_notes.md — bumped 0.2.00.2.1 and added a changelog entry (matches the repo's version-field + release_notes.md convention; repo has no git tags).

Test plan

  • pytest --disable-socket unit suite — 90 passed (no code path changed; docstrings/version only).
  • python -c "ast.parse(...)"tools.py compiles.
  • ruff check — net +3 E501 (long-line) only, matching this file's prevailing one-line-per-param docstring style (177 pre-existing E501; repo lint baseline is already non-clean and ungated). No new rule classes introduced.
  • Integration tests (make integration_tests) require a live SCRAPINGBEE_API_KEY and the server Auto-Mode flag enabled — run when the server feature ships (this is why the PR stays draft).
  • Manual: agent picks mode='auto' for an unknown site and Spb-auto-cost reflects the winning tier.

🤖 Generated with Claude Code

Auto-Mode (mode=auto) lets the ScrapingBee API pick the cheapest
scraping config that succeeds (cheapest->expensive, stops at first
success, charges only the winning config). The tools already pass
arbitrary params straight through to the API, so this is a docs-only
change so an LLM agent will actually choose it:

- langchain_scrapingbee/tools.py: add `mode` and `max_cost` to
  ScrapeUrlTool's documented/allowed params (SUPPORTED PARAMS list +
  prose hint + ScrapeUrlInput.params example). Documents accurately:
  mode='auto' is GET-only, cheapest-first, charged only for the winning
  config (0 on full failure), reports credits in the Spb-auto-cost
  header, and must not be combined with render_js/premium_proxy/
  stealth_proxy/transparent_status_code; max_cost (int >= 1) caps spend
  and requires mode='auto'.
- README.md: add an Auto-Mode usage example mirroring the existing ones.
- pyproject.toml / release_notes.md: bump 0.2.0 -> 0.2.1 + changelog
  entry (matches repo's version-field + release_notes convention).

No request-code or architecture change; no validation added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kostas-jakeliunas-sb kostas-jakeliunas-sb marked this pull request as ready for review June 30, 2026 12:58
@kostas-jakeliunas-sb kostas-jakeliunas-sb changed the title [DRAFT — DO NOT MERGE] [SCR-383] Auto-Mode (mode=auto) support [SCR-383] Auto-Mode (mode=auto) support Jun 30, 2026
@kostas-jakeliunas-sb kostas-jakeliunas-sb self-assigned this Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants