Skip to content

test: validate QWED Security v1.1 scanner across all engines #214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Rahul Dass (rahuldass19) wants to merge 9 commits into from
Closed

test: validate QWED Security v1.1 scanner across all engines #214

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4ce729a
test: add demo files for QWED Security v1.1 validation
rahuldass19 Jul 20, 2026
c083d33
test: add taint sink for v1.2 validation
rahuldass19 Jul 20, 2026
0e4dd90
test: bump version string for v1.3 re-check
rahuldass19 Jul 20, 2026
ada4162
test: bump version for v1.4 re-check
rahuldass19 Jul 21, 2026
50d8d4f
Merge branch 'main' of https://github.com/QWED-AI/qwed-verification i…
rahuldass19 Jul 23, 2026
837094c
test: trigger QWED Security Phase 3 pipeline scan
rahuldass19 Jul 27, 2026
15e917f
test: add runtime secret to verify HARD BLOCK on PR #214
rahuldass19 Jul 27, 2026
9dd27e6
test: add runtime secret in RUNTIME_CODE file to verify HARD BLOCK
rahuldass19 Jul 27, 2026
840b0fb
test: re-check runtime secret regex
rahuldass19 Jul 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .qwed.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
treat_unknown_as_block: true
ignore_patterns_in_tests: true
Comment on lines +1 to +2

@gitar-bot gitar-bot Bot Jul 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Security: New .qwed.yml disables pattern scanning in test files

ignore_patterns_in_tests: true instructs the security scanner to skip pattern-based scans for files considered tests. In a security-verification product this is a meaningful regression: secrets, dangerous sinks, or malicious code placed in any file matching a test path would evade detection. Note the accompanying test file is named qwed_security_test.py, which would fall under this exclusion. Recommend removing this key (or scoping it very narrowly) so scanning stays fail-closed for all paths.

Was this helpful? React with 👍 / 👎

Comment on lines +1 to +2

@gitar-bot gitar-bot Bot Jul 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: Unknown config keys in .qwed.yml are not consumed by codebase

Neither treat_unknown_as_block nor ignore_patterns_in_tests appears anywhere in the repository's source (config loaders, validators, or models). As written these keys are silently ignored by this repo's code, so the file either has no effect here or is intended to be interpreted only by an external tool. Confirm the intended consumer and the exact schema; a typo'd or unrecognized key gives a false sense that a policy is in force when it is not.

Was this helpful? React with 👍 / 👎

@greptile-apps greptile-apps Bot Jul 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Validation Fixture May Be Ignored

If test-file detection uses the *_test.py naming convention, this setting suppresses pattern findings for examples/qwed_security_test.py. The expected credential advisory would then never be produced, so the fixture would not validate the claimed pattern-scan behavior.

@sentry sentry Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The ignore_patterns_in_tests configuration in .qwed.yml is not implemented in the scanner, failing to suppress a real eval() vulnerability in a new test file.
Severity: HIGH

Suggested Fix

Implement the logic in action_entrypoint.py to parse .qwed.yml and correctly apply the ignore_patterns_in_tests configuration. Alternatively, remove the non-functional configuration and the test file containing the eval(input(...)) pattern to avoid the false sense of security.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: .qwed.yml#L2

Potential issue: The configuration file `.qwed.yml` introduces a key
`ignore_patterns_in_tests: true` intended to suppress security findings in test files.
However, the scanner logic in `action_entrypoint.py` does not read this configuration
file. The scanner will still scan all Python files, including the new test file
`examples/qwed_security_test.py`, and flag the dangerous `eval(input(...))` pattern it
contains. This discrepancy means the configuration is non-functional and creates a false
sense of security, as the vulnerability will be detected, contrary to the pull request's
stated intent.

Also affects:

  • examples/qwed_security_test.py:11~14

3 changes: 3 additions & 0 deletions examples/demo_widget.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
const display = (text) => {
document.getElementById("out").textContent = text;
};
8 changes: 8 additions & 0 deletions examples/qwed_security_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
"""Demonstrates QWED Security scanning capabilities."""
Comment thread
greptile-apps[bot] marked this conversation as resolved.
Outdated

API_KEY = "QWED_TEST_DEMO_VALUE"

Check failure on line 3 in examples/qwed_security_test.py

View check run for this annotation

QWED Security / QWED Security

QWED: pattern_scan 

Hardcoded credential-like material detected. Context=TEST_CODE. Decision reason: Pattern remains enforceable even in test code.

@coderabbitai coderabbitai Bot Jul 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Remove the credential-shaped test constant.

Line 3 is classified as hardcoded credential-like material, and the QWED pipeline blocks it rather than emitting the expected advisory. Use a clearly non-credential test marker or move this case into a dedicated scanner fixture with an explicit test contract; do not weaken global scanning.

As per path instructions, enforcement must remain deterministic and fail-closed.

🧰 Tools
🪛 GitHub Check: QWED Security

[error] 3-3: QWED Blocked finding: Hardcoded credential-like material detected (context: TEST_CODE).

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/qwed_security_test.py` at line 3, Replace the credential-shaped
API_KEY value in the test fixture with a clearly non-credential marker, or move
the scenario into a dedicated scanner fixture with an explicit test contract.
Preserve deterministic, fail-closed global scanning and do not weaken its
enforcement.

Sources: Path instructions, Linters/SAST tools, Pipeline failures



def run_demo():
user_data = input("Enter expression: ")
print(f"You entered: {user_data}")
Loading