Skip to content

Add bounded DAST external evaluation - #1

Draft
ZENLIK02 wants to merge 1 commit into
mainfrom
agent/update-bounded-dast-evaluation
Draft

Add bounded DAST external evaluation#1
ZENLIK02 wants to merge 1 commit into
mainfrom
agent/update-bounded-dast-evaluation

Conversation

@ZENLIK02

Copy link
Copy Markdown
Owner

What changed

  • Adds the frozen bounded-DAST validator and its protocol documentation.
  • Integrates bounded validation with the CLI/config/evaluation path and supporting tests.
  • Adds the full OWASP BenchmarkJava eligibility/evaluation harness and the adversarial safety-gate harness.
  • Adds measured JSON/CSV/Markdown results, Wilson intervals, counterfactual metrics, repeatability checks, and the SHA-256 manifest.
  • Adds updated paper sources, figures, tables, and IEEE-ready report-generation assets.

Why

This publishes the tested Heimdall revision and its reproducibility artifacts so the bounded post-SAST validation claims can be audited against the exact implementation and measured outputs.

Impact

The repository now contains the loopback-only, one-request-per-alert validation path plus the scripts and evidence needed to reproduce the controlled and external evaluations. Existing remote-only demo summaries were preserved.

Validation

  • 61 passed, 18 subtests passed
  • Frozen validator SHA-256 before/after: 2bbb3b1086cdfe249b50856179854a325be870ce43d312788fb3f1b063d5ef31
  • Frozen external config SHA-256 before/after: b47cceefb35bd12878adb1ee9e9474f74bde862eb37004e6dfcc2b3ff08182b3
  • All entries in SHA256SUMS.txt verified
  • No real credentials detected; matched key strings are fixtures/placeholders or environment-variable names

@github-actions

Copy link
Copy Markdown

Heimdall CI/CD Summary

Pipeline status: PASSED

Metric Count
Total Semgrep findings 24
Total validated findings 24
Confirmed True Positives 0
False Positives 11
Needs Review 13
High/Critical confirmed vulnerabilities 0

Findings

Rule ID Severity File Line Heimdall Decision Evidence Recommended Action
yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag medium .github/workflows/heimdall-devsecops.yml 17 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag medium .github/workflows/heimdall-devsecops.yml 20 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag medium .github/workflows/heimdall-devsecops.yml 47 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag medium .github/workflows/heimdall.yml 16 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag medium .github/workflows/heimdall.yml 19 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
yaml.github-actions.security.github-actions-mutable-action-tag.github-actions-mutable-action-tag medium .github/workflows/heimdall.yml 68 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
dockerfile.security.missing-user.missing-user high Dockerfile 13 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected medium heimdall/pipeline/dast_executor.py 90 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
python.flask.security.audit.directly-returned-format-string.directly-returned-format-string medium local_lab/vulnerable_app/app.py 47 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.django.security.injection.raw-html-format.raw-html-format medium local_lab/vulnerable_app/app.py 47 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.flask.security.injection.raw-html-concat.raw-html-format medium local_lab/vulnerable_app/app.py 47 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.flask.security.audit.directly-returned-format-string.directly-returned-format-string medium local_lab/vulnerable_app/app.py 53 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.django.security.injection.raw-html-format.raw-html-format medium local_lab/vulnerable_app/app.py 53 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.flask.security.injection.raw-html-concat.raw-html-format medium local_lab/vulnerable_app/app.py 53 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.lang.security.unverified-ssl-context.unverified-ssl-context high scripts/external_full_evaluation.py 330 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected medium scripts/post_pr_comment.py 39 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
python.lang.security.unverified-ssl-context.unverified-ssl-context high scripts/run_external_full_with_benchmark.py 36 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected medium scripts/run_external_full_with_benchmark.py 38 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
python.lang.security.unverified-ssl-context.unverified-ssl-context high scripts/safety_gate_evaluation.py 655 Needs Review dry-run generic response Collect missing context and repeat validation in an authorized test environment.
python.flask.security.audit.directly-returned-format-string.directly-returned-format-string medium test_apps/flask_vulnerable_app/app.py 25 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.django.security.injection.raw-html-format.raw-html-format medium test_apps/flask_vulnerable_app/app.py 25 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.flask.security.injection.raw-html-concat.raw-html-format medium test_apps/flask_vulnerable_app/app.py 25 False Positive Controlled marker appears escaped or inert. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.django.security.injection.tainted-sql-string.tainted-sql-string high test_apps/flask_vulnerable_app/app.py 31 False Positive Input rejected or authentication failed without SQL error. Document the defensive control and suppress or tune the SAST rule if appropriate.
python.flask.security.injection.tainted-sql-string.tainted-sql-string high test_apps/flask_vulnerable_app/app.py 31 False Positive Input rejected or authentication failed without SQL error. Document the defensive control and suppress or tune the SAST rule if appropriate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant