Skip to content

Test: added benign SQL and script regression tests with narrow payload fix - #133

Open
aaishii07 wants to merge 7 commits into
0-Shimanshu:mainfrom
aaishii07:test/benign-sql-html-regression
Open

Test: added benign SQL and script regression tests with narrow payload fix#133
aaishii07 wants to merge 7 commits into
0-Shimanshu:mainfrom
aaishii07:test/benign-sql-html-regression

Conversation

@aaishii07

Copy link
Copy Markdown

Summary

Added two regression tests for harmless discussion-style SQL and HTML text that was being incorrectly flagged as hostile. Added a narrow fix to the payload signal to prevent false positives on these cases.

Why

The payload signal was scoring sentences like "How do I write SELECT * FROM users in a tutorial?" at 0.72, treating natural language discussion as SQL injection. Same for <script>.

Scope

  • bug fix
  • tests
  • signal change

Verification

pytest tests/test_benign_matrix.py -v
pytest --ignore=tests/test_cli.py -q

Notes:

  • I ran the relevant tests locally

Signal-specific notes

  • soft signal (trackB)

Source: select_from pattern and vendor detect_xss were triggering on discussion text.

Benign case checked: "How do I write SELECT * FROM users in a tutorial?" → 0.0, "How do I print <script> literally in docs?" → 0.0

TUI notes

  • not applicable

Local-only file check

  • I did not accidentally stage local-only files such as adiuvare.yaml or local conftest.py changes

Related issue

Closes #6

@github-actions github-actions Bot added the enhancement New feature or request label May 25, 2026
@Wahid7852

Copy link
Copy Markdown
Collaborator

hey @aaishii07, please add tests showing actual SQLi/XSS payloads with conversational framing still score correctly, for example:

natural language + UNION SELECT payloads
natural language + executable <script> payloads
mixed benign/malicious content in the same input

otherwise its quite hard to verify this didnt unintentionally weaken the payload signal logic while reducing false positives.

@Wahid7852

Copy link
Copy Markdown
Collaborator

also a few concerns:

  • why are .egg-info generated packaging files being committed in this PR at all? these should not be part of a signal regression fix.

  • _is_discussion_context() is extremely broad right now. matching words like how, show, write, docs, etc. as a bypass condition feels very fragile for a security signal.

  • this also only exempts select_from and script_tag, which feels pattern-specific instead of solving the actual context problem properly.

  • the PR still doesnt include adversarial regression coverage proving mixed conversational + malicious payloads continue to score correctly.

@aaishii07

aaishii07 commented May 28, 2026

Copy link
Copy Markdown
Author

Hi @Wahid7852 ! I've addressed all the concerns:

  • Removed egg-info files and added to .gitignore.
  • Replaced the broad keyword list in "_is_discussion_context" with a stricter regex that requires a question word at the start and a "?" at the end – so words like "how", "show", and "write" alone can't bypass detection.
  • Added "_is_executable_xss" to distinguish real payloads from discussion text, so "<script>alert(1)</script>" still scores high even inside a question.
  • Added 3 adversarial regression tests showing actual SQLi/XSS payloads with conversational framing. All still score correctly.

Comment thread adiuvare/signals/patterns.py Outdated
Comment thread adiuvare/signals/patterns.py Outdated
Comment thread adiuvare/signals/payload.py Outdated
Comment thread adiuvare/signals/payload.py Outdated
Comment thread tests/test_payload.py Outdated
@Wahid7852

Copy link
Copy Markdown
Collaborator

Good effort. Once the merge conflicts are resolved and the _EXECUTABLE_SCRIPT_RE bypass is tightened up this will be in a good shape to merge.

@0-Shimanshu

Copy link
Copy Markdown
Owner

@aaishii07

…on logic in one file, fixed executable script detection, made threshold consistent
@aaishii07

Copy link
Copy Markdown
Author

@Wahid7852 I have resolved all the concerns. If any more do rise up, please let me know!

@Wahid7852 Wahid7852 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

everything else looks good, merge the conflicts and should be ready to merge.
cc @0-Shimanshu, check it out once

Comment thread adiuvare/signals/payload.py Outdated
@0-Shimanshu

Copy link
Copy Markdown
Owner

@aaishii07 conflict

@aaishii07
aaishii07 requested a review from Wahid7852 June 4, 2026 22:30
@0-Shimanshu
0-Shimanshu self-requested a review June 5, 2026 04:35
@0-Shimanshu

Copy link
Copy Markdown
Owner

@aaishii07

@Wahid7852

Copy link
Copy Markdown
Collaborator

@aaishii07 will close this if no activity within 4-5 days

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: add regression coverage for harmless SQL and HTML discussion text

3 participants