feat(workflows): add api_discover workflow and api scan#1268
Conversation
Add an API-focused discovery workflow and a composed API security scan, reusing existing workflows rather than duplicating their logic. - secator/configs/workflows/api_discover.yaml: API endpoint discovery via katana (crawl) + optional ffuf route brute-forcing, then httpx to verify discovered endpoints. Endpoint discovery only. - secator/configs/scans/api.yaml: chains api_discover -> url_params_fuzz -> url_vuln (param fuzzing + vuln/secrets scanning), mirroring the url scan. - config.py: add the "apiroutes" wordlist template pointing at Assetnote's maintained HTTP Archive API routes dataset (the same real-world route data kiterunner's -A apiroutes relied on), used by ffuf. - tests: add api_discover/api entries to integration inputs and outputs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughAdds a new ChangesAPI Discovery Feature
Estimated code review effort: 2 (Simple) | ~10 minutes Sequence Diagram(s)sequenceDiagram
participant Scanner as "api scan"
participant Discover as "api_discover workflow"
participant Katana
participant Ffuf
participant Wafw00f
participant Httpx
participant Fuzz as "url_params_fuzz"
participant Vuln as "url_vuln"
Scanner->>Discover: run(url)
Discover->>Katana: crawl endpoints
Discover->>Ffuf: brute-force routes (apiroutes, if fuzz)
Discover->>Wafw00f: fingerprint WAF (if waf)
Discover->>Httpx: probe endpoints (unverified urls)
Discover-->>Scanner: verified urls
Scanner->>Fuzz: target verified urls
Scanner->>Vuln: target verified urls
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
secator/config.py (1)
171-172: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winHardcoded dated URL will never actually refresh monthly.
Assetnote regenerates and publishes a new dated file each month, but the old dated files stay live indefinitely on the CDN rather than being replaced. Pinning to
httparchive_apiroutes_2026_06_27.txtmeans this wordlist is frozen at that exact snapshot forever — it won't 404, but it also won't ever pick up new data despite the "regenerated monthly" comment implying freshness.Consider tracking this with a periodic bump reminder/CI check, or investigating whether Assetnote exposes a stable "latest" alias.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@secator/config.py` around lines 171 - 172, The apiroutes entry in config is pinned to a dated Assetnote CDN file, so it will never auto-refresh despite the monthly-regenerated comment. Update the config handling around the apiroutes constant to avoid hardcoding a snapshot URL; either switch to a stable “latest” alias if one exists or add a clear periodic update mechanism/check tied to the apiroutes source so the dataset stays current. Use the apiroutes key in secator/config.py to locate the mapping.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@secator/config.py`:
- Around line 171-172: The apiroutes entry in config is pinned to a dated
Assetnote CDN file, so it will never auto-refresh despite the
monthly-regenerated comment. Update the config handling around the apiroutes
constant to avoid hardcoding a snapshot URL; either switch to a stable “latest”
alias if one exists or add a clear periodic update mechanism/check tied to the
apiroutes source so the dataset stays current. Use the apiroutes key in
secator/config.py to locate the mapping.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: ea243ad9-a428-4ef0-bba2-4509cf7a054b
📒 Files selected for processing (5)
secator/config.pysecator/configs/scans/api.yamlsecator/configs/workflows/api_discover.yamltests/integration/inputs.pytests/integration/outputs.py
When api_discover finds an exposed OpenAPI/Swagger spec, the new --spec option routes it to nuclei with input-mode=openapi + DAST fuzzing, which parses the spec and fuzzes every documented endpoint with the correct method/parameters. This recovers the contextual API testing kiterunner used to provide, using a maintained tool already integrated in secator (no new dependency). - nuclei task: add a `dast` flag (-dast) to enable fuzzing templates. - api_discover.yaml: add --spec option and a nuclei step targeting discovered spec URLs (openapi/swagger/api-docs), gated behind --spec. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary Adds the **OpenAPI-spec handoff** to `api_discover`: when discovery finds an exposed OpenAPI/Swagger spec, hand it off to **nuclei** (already integrated) to parse the spec and DAST-fuzz every documented endpoint. This recovers the *contextual* API testing that kiterunner used to provide (correct method + parameters per route) — but with a **maintained tool already in secator**, so **no new dependency**. > Stacked on #1268 (`workflow-api`). Base will be retargeted to `main` once #1268 merges. ## What's included - **`secator/tasks/nuclei.py`** — new `dast` flag (`-dast`) to enable nuclei's fuzzing templates (required to fuzz OpenAPI endpoints). Defaults to off; no change to existing behavior. - **`secator/configs/workflows/api_discover.yaml`** — new `--spec` option and a `nuclei` step that targets discovered spec URLs (`openapi` / `swagger` / `api-docs`) with `input_mode=openapi` + `dast`. Gated behind `--spec` (off by default), so pure discovery is unchanged. ## How it works ``` api_discover finds /openapi.json ──(--spec)──► nuclei -u <spec-url> -input-mode openapi -dast (katana / ffuf --fuzz) parses the spec, fuzzes every documented endpoint ``` Typical usage: `secator w api_discover <url> --fuzz --spec` (fuzz surfaces the spec, --spec scans it). ## Design notes - **Why nuclei and not a new tool?** nuclei natively supports OpenAPI/Swagger input (`-input-mode openapi`, its SpecDownloader fetches a remote spec URL) and DAST fuzzing (`-dast`). Adding schemathesis/CATS/apifuzzer would either duplicate this or drag in an unmaintained tool / Java runtime — against secator's design principle #1. - **Separation of concerns:** the step is optional and flag-gated, consistent with how `url_crawl` gates its optional trufflehog step. ## Testing - Command generation verified: `nuclei ... -input-mode openapi -dast` emitted correctly. - `api_discover` loads and builds with `--spec` (tasks: katana, ffuf, wafw00f, httpx, nuclei). - nuclei unit test passes (no regression from the `dast` flag). - `flake8 secator/tasks/nuclei.py` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Summary
Adds an API-focused discovery workflow and a composed API security scan, reusing existing workflows rather than duplicating logic (per secator's design philosophy).
What's included
secator/configs/workflows/api_discover.yaml— API endpoint discovery only:katana(crawl, JS-aware) discovers referenced endpoints.ffuf(opt-in via--fuzz) brute-forces API routes using theapirouteswordlist.wafw00f(opt-in via--waf) fingerprints the WAF.httpxprobes discovered endpoints to verify they're live and fingerprint tech.secator/configs/scans/api.yaml— chainsapi_discover→url_params_fuzz→url_vuln, mirroring the existingurlscan. Parameter fuzzing, vulnerability scanning and secrets hunting are handled by the existing workflows — no duplication.config.py— newapirouteswordlist template pointing at Assetnote's HTTP Archive API routes dataset, regenerated monthly (the same real-world route data kiterunner's-A apiroutesrelied on).api_discover/apientries in integrationinputs.py/outputs.py.Design notes
ffuf.httparchive_apiroutesfile is just a stale 2022 copy of the Assetnote dataset we now use fresh; their param/payload lists belong to the downstreamurl_params_fuzz/url_vulnstages which already have curated sources.apifuzzer/openapi-fuzzer, spec-driven):api_discoverfinds exposed OpenAPI/Swagger specs that a spec-driven fuzzer could later consume.Testing
api_discovertasks: katana, ffuf, wafw00f, httpx;apiscan workflows: api_discover, url_params_fuzz, url_vuln).Workflow/Scanobjects build without error.flake8 secator/config.pyclean.--workflows api_discover,--scans api) require the live lab + tools (run in CI).🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes