BugsJS bug dataset

This repository contains the data files and the utility framework of the BugsJS benchmark.

Projects

This benchmark includes bugs from the following 10 projects:

• Bower
• Eslint
• Express
• Hessian.js
• Hexo
• Karma
• Mongoose
• Node-redis
• Pencilblue
• Shields

Note

The tests of the Node-redis project require ipv6 support.

The per-test coverage data of the ESlint project is located in a separate storage due to its size (3GB).

ESlint coverage data

Files in the repository

In the Projects directory, there is a subdirectory for each project which contains the following files:

• [project]_bugs.csv: pre-calculated information about the tests of each bug
• [project]_commands.csv: the necessary test runner commands for each bug
• [project]_issues.bin: the bug reports of the subject project save in the Google's Protocol Buffers format (for the structure see the project.proto file)

Commands

The framework's command-line interface includes the following commands:

• info: Prints out information about a given bug.
• checkout: Checks-out the source code for a given bug.
• test: Runs all tests for a given bug and measures the test coverage.
• per-test: Runs each test individually and measures the per-test coverage for a given bug.

For the checkout, test, and per-test commands, the user can specify the desired code revision:

• buggy: The parent commit of the revision in which the bug was fixed.
• fixed-only-test-change: A revision containing only the tests introduced in the bug fixing commit, applied to the buggy revision.
• fixed: A revision containing both the cleaned fix and the newly added tests.

Example command:

python3 main.py -p Bower -b 1 -t checkout -v fixed -o output/