fix(backend): warn when knowledgebase feeds are malformed (#1837) - #2
Draft
NaitikVerma6776 wants to merge 719 commits into
Draft
fix(backend): warn when knowledgebase feeds are malformed (#1837)#2NaitikVerma6776 wants to merge 719 commits into
NaitikVerma6776 wants to merge 719 commits into
Conversation
utksh1#1398 # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.
- New reportTemplates.ts service with ReportTemplate type, three built-in templates (executive, technical, compliance), and render/preview/export. - ReportTemplatePicker.tsx slide-over component with type filtering, inline preview, and .md export. - Integrate Templates button into Reports.tsx report cards. - 21 unit tests covering template lifecycle, edge cases, and output.
…tksh1#1547) * Resolves issue-utksh1#1413 * test: remove trailing whitespace
…1546) Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
…h1#1542) Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
Co-authored-by: tmdeveloper007 <tmdeveloper007@users.noreply.github.com>
…ksh1#1527) * Resolves issue-utksh1#1427 * Removes white spaces.
The saved_views_router was registered in main.py without the require_api_key dependency, allowing unauthenticated access to all CRUD endpoints for saved views. This is an authentication bypass that exposes filter configurations and enables data tampering. Add Depends(require_api_key) to the router definition, matching the pattern used by the main API router in routes.py.
…h1#2036) The application defaulted to debug=True, causing: 1. Full Python tracebacks returned as HTML on unhandled exceptions 2. OpenAPI docs always exposed at /docs and /openapi.json 3. Uvicorn hot-reload enabled in production Fix: - Change debug default from True to False in config.py - Conditionally disable docs/redoc/openapi endpoints when debug=False - Conditionally skip /api/docs redirect routes when debug=False - Update .env.example to reflect safe default (SECUSCAN_DEBUG=false)
Add a dedicated security CI/CD pipeline that runs on push/PR to main. Catches the classes of vulnerabilities identified in issues utksh1#2035 and utksh1#2036: - Bandit Python security linting (high-severity gate) - Route authentication protection checks - Debug mode default validation - Hardcoded secret detection - Environment configuration validation - Dependency vulnerability auditing This provides continuous security regression testing to prevent similar vulnerabilities from being introduced in future PRs.
Export getApiKey from api.ts and update useSavedViews apiFetch to include X-Api-Key header and credentials:'include', matching the authenticated request pattern used by the rest of the API client. Without this, saved-view requests silently fail with 401 in deployments where API_BASE is absolute, causing the feature to fall back to localStorage even after login.
The dev workflow in start.sh advertises /docs, /redoc, and /openapi.json but never sets SECUSCAN_DEBUG. With the new default of debug=False, these routes 404 unless developers manually set the env var. Export SECUSCAN_DEBUG=true before launching uvicorn so the dev server has full API docs and hot-reload.
Fix two issues flagged by the bot review: 1. Debug guard check: The original logic used content.split(line) which incorrectly sliced strings. Replace with proper backward scan that checks the 10 lines above traceback.format_exc for an if-settings.debug guard. 2. Secret scan regex: Replace character class quoting with hex-escaped patterns to avoid YAML/Python shell quoting conflicts that caused SyntaxError. Also simplify the POSTGRES_PASSWORD pattern. All f-string interpolations replaced with .format() to avoid nested quote escaping issues in YAML run blocks.
…-parser-validation fix(plugins): enforce strict validation and integrity checks for missing custom parser utksh1#1812
…to ID generation Addresses Bandit HIGH severity findings: - finding_intelligence.py:75 - stable ID generation - platform_resources.py:29 - asset ID generation These SHA1 usages are for deterministic hashing to create stable IDs, not for cryptographic security purposes. Adding usedforsecurity=False clarifies the intent and resolves the security linter warnings.
The debug default was changed from True to False in the security fix. Update the test to match the new secure default.
The saved_views_router now has require_api_key dependency. Override it in tests to bypass authentication for unit testing.
Add shared time_utils helpers and use timezone-aware UTC with an explicit offset for generated_at and discovered_at across reports, findings API responses, and report generation. Closes utksh1#1882
Default to_utc_iso to timespec=auto so finding intelligence tests can compare against datetime.now(UTC). Update TLS verification mocks for crawler client.stream() and stub crawl_target in API scanner tests.
…idable _init_default_policies() built the entire network denylist from the single Pydantic field settings.network_denylist. Pydantic replaces (rather than merges) a list field's default when SECUSCAN_NETWORK_DENYLIST is set via env var, so any operator adding even one custom denylist entry silently dropped the built-in protection for cloud metadata (169.254.169.254), loopback, RFC1918/CGNAT ranges, and IPv6 link-local/ULA space -- reopening SSRF to the metadata endpoint despite the code comment claiming the denylist was 'always enforced'. Fix: move those ranges into a new MANDATORY_DENYLIST module constant that is not read from settings and is applied unconditionally in _init_default_policies before any operator-configured entries. The operator-facing network_denylist setting is now purely additive. Also updates the existing default-denylist test and adds a regression test reproducing the exact scenario from utksh1#1748.
…ne-standardize-9bb6 fix(backend): standardize timezone handling to UTC ISO-8601
…t-metadata-ssrf Fix utksh1#1748: make cloud-metadata/private-range denylist non-overridable
Fix: add auth and owner isolation to saved views API (closes utksh1#1743)
Cover the scapy_recon plugin parser.py with targeted behavioural tests: - Metadata contract: file existence, valid JSON, required fields, engine binary, target/type field declarations - ARP output: host count, IP+MAC extraction, finding keys, category, severity, description content, metadata consistency, remediation - ICMP output: host count, IP extraction, Unknown-MAC default - Single-host edge case: IP+MAC in result and description - Malformed/empty input: empty string, whitespace-only, no UP: lines, mixed noise lines, malformed UP: lines, missing MAC separator No changes to backend source; test file only.
Surface warning logs with the feed file name when a JSON feed has a non-object top-level value, fails to parse, or contains CPE entries that are not lists, instead of silently dropping them (utksh1#1837).
Use a real-auth client fixture for saved-views 401 tests so they are not bypassed by the require_api_key override. Upgrade postcss to >=8.5.18 to clear GHSA-r28c-9q8g-f849 blocking frontend CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1837 — knowledgebase previously skipped malformed feed files with no useful signal, so a misconfigured feed could yield zero CVEs silently.
Changes
Test plan
testing/backend/unit/test_knowledgebase.py(malformed top-level, invalid JSON, malformed CPE entry)backend-unitgreen