BugsJS bug dataset

This repository contains the data files and the utility framework of the BugsJS benchmark.

Projects

This benchmark includes bugs from the following 10 projects:

The tests of the Node-redis project require ipv6 support.

The per-test coverage data of the ESlint project is located in a separate storage due to its size (3GB).

In the Projects directory, there is a subdirectory for each project which contains the following files:

[project]_bugs.csv: pre-calculated information about the tests of each bug
[project]_commands.csv: the necessary test runner commands for each bug
[project]_issues.bin: the bug reports of the subject project save in the Google's Protocol Buffers format (for the structure see the project.proto file)

The framework's command-line interface includes the following commands:

info: Prints out information about a given bug.
checkout: Checks-out the source code for a given bug.
test: Runs all tests for a given bug and measures the test coverage.
per-test: Runs each test individually and measures the per-test coverage for a given bug.

For the checkout, test, and per-test commands, the user can specify the desired code revision:

buggy: The parent commit of the revision in which the bug was fixed.
fixed-only-test-change: A revision containing only the tests introduced in the bug fixing commit, applied to the buggy revision.
fixed: A revision containing both the cleaned fix and the newly added tests.

Example command:

python3 main.py -p Bower -b 1 -t checkout -v fixed -o output/