feat(benchmarking): added bugs to QA benchmarking [FOU-80] #1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📋 Summary
This PR introduces a QA Benchmark Seed containing a set of intentional, toggleable bugs for use in QA benchmarking and performance evaluation. These bugs are safe, isolated, and can be enabled/disabled locally or via environment variables without impacting production.
🐛 Included Demo Bugs
🔧 Usage
Each bug is documented with:
Enable/Disable instructions
Reproduction steps
Modified files
Expected incorrect behavior
These can be found in
qa-benchmark/along withtickets.csv, which lists the bugs as realistic QA tickets.🛡️ Safety
Bugs are opt-in only (disabled by default)
No side effects without explicitly enabling the respective flags
No changes to production behavior unless deliberately toggled