Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
All Files Linted
This pull request is the result of executing a linting script on all C/C++, Verilog/SystemVerilog, Python and VHDL files in the ESP repository as of May 29, 2024.
For more information on how linting happened, refer to the linting scripts in the PR titled: "Linting Scripts" or the documentation below (which can also be found under utils/scripts/actions-pipeline/README.md in the
regression-flow
branch.)ESP Linting and Regression Testing Workflows
The scripts included in this directory are used as part of a GitHub Actions workflow triggered on PRs to the dev, main or master branches.
Scripts under the /code-format directory are used for linting purposes, while the other scripts are used for regression testing.
Scripts
Code Formatting
There are two scripts under ./code-format that can be used by contributors to check their code for formatting violations, as well as to format code in place. These scripts could also be used as part of a Git hook or a GitHub Actions workflow.
The scripts support formatting for the following languages: C/C++, Verilog/SystemVerilog, VHDL, and Python.
Format Newly-Modified Files
format_modified.sh is a bash script that examines the Git status of the remote repository to identify files that the user has recently modified, added, or deleted. It then determines the file extension type (e.g., C/C++, VHDL, SystemVerilog/Verilog, or Python) and invokes an open-source linting tool for that language with the filename and other necessary arguments. The script offers flexibility through various flags, allowing users to report programming violations, as well as, fix formatting in-place. This tool plays a crucial role in maintaining ESP's readability, consistency, and bug-free nature.
Format ESP Repo
The format_modified.sh script is useful for formatting new modifications to existing code. However, this implies that the code in the repository is already properly structured to begin with. In reality, our ESP repository lacks uniformity. To introduce formatting into the existing code, we have provided another script called format_repo.sh which traverses through all directories in the ESP repository recursively, skipping over any directories that are Git submodules owned by another organization. The script identifies the file extension of each file within the current directory and formats it in-place by calling the appropriate open-source linter.
format_repo.sh is intended for one-time use. Additionally, it can significantly impact Git blame, potentially attributing all changes to a single contributor (whoever ran the script). To mitigate this, changes made by format_repo.sh should be committed separately and excluded from the blame.
Git Blame
After processing the Pull Request, it's important to consider what commits we want to exclude from the Git Blame. In particular, it should be any commits that have modified/linted tracked files in bulk.
To make this process easier, I have committed all the linting changes in 4 different commits in the
all-linted-files
branch.These commit ids can be added to a file which can be excluded from the blame. This tutorial is helpful in the process of excluding from the Git Blame.
Regression Testing
Get Modified Accelerators
This script accepts a configuration file called accelerators.json and uses the repository's Git history to determine modified accelerators. The configuration file is simply a JSON file where ESP members can specify what accelerators they wish to test. Imagine that a user makes modifications to many accelerators in a single Pull Request. We may only care about testing a subset of these at a given time.
By default the get_modified_accelerators.sh scripts can run end-to-end tests with each modified accelerator in a 2x2 SoC, but having the configuration file adds extra flexibility to define/override which accelerators are to be tested.
The script compares the modified files (as specified in the Git history) to the accelerators in the configuration file. It then returns a list of the overlap of accelerators that should be tested, along with references to their HLS and behavioral
make
commands (ie.make cholesky_stratus-hls
ormake cholesky_stratus-beh
).Run Simulations
The Run Simulations script accepts a list of testable accelerators via a bash list, and runs HLS for each one. The script first assesses how many accelerators have been scheduled for synthesis.
We consider that in reality a user may only modify one or two accelerators at a time, but designed this project such that if many accelerators are modified in one Pull Request
the program can balance them all effectively across different CPUs on the SOCP server and execute them in parallel. The script iterates through the list of accelerators
and starts and HLS run for each one, asynchronously. Then, it waits for all HLS jobs to return, logging the results for each job as they complete. Once all jobs have
completed, the script exits with success or error code.
Run ESP Config
After the Run Simulations scripts executes, the Run ESP Config script should be executed. Run ESP Config is a bash script that completes the bitstream generation,
FPGA programming, and the test program execution on one of the ESP Xilinx FPGAs. Run ESP Config first looks through the tech/acc folder in the working repository.
This directory has all the work folders for the completed HLS jobs. The script iterates through all the successful accelerator HLS jobs and creates an ESP 2x2 SoC configuration
with that accelerator in the accelerator tile. The script then proceeds to run
make esp-config
andmake vivado-syn
to generate the HDL and bitstream for thefull SoC configuration. If this succeeds, the script will continue to run
make fpga-program
to program the generated bitstream on the FPGA, and execute a testprogram using
make fpga-run
. The script opens up a minicom connection to the FPGA simultaneously to runningmake fpga-run
. The results of the test programare read out from the minicom to a log file for users to refer to after completion.
If more than one accelerator has been modified, the script will start a new end-to-end test by creating the ESP configuration, generating HDL, generating the bitstream,
configuring the FPGA and running the test program with the next accelerator, and so on.
Setting Up Pre-push Hook
To ensure code formatting compliance, we provide a pre-push Git hook that runs before you push changes to the repository. Follow these steps to set it up:
Now, every time you try to push changes to the repository, the pre-push hook will run automatically to check code formatting compliance.
Setting Up Self-Hosted GitHub Runners
GitHub Actions is a tool integrated into the ESP GitHub repository that consists of YAML workflows that can be triggered by different git operations such as push or pull request.
Normally, GitHub Actions is executed on a GitHub-hosted server and can be operated at no-cost with a limit on the number of workflow runs.
This is useful for many applications but for ESP, we have a variety of licensed tools that are used for HLS and other procedures which are not easily accessed via GitHub-hosted servers.
Instead, it makes sense for ESP to run regression workflows on the SOCP servers for more control and flexibility. For this, GitHub offers self-hosted runners which are easily configurable
via a downloadable executable program. The resources below explain how a self-hosted runner can be created and configured for ESP.
For testing, I set up the actions-runner program under the ma4107 user in socp01, then executed the
run.sh
script within the project to enable the tool to listen for GitHub Actionsjobs.
Resources
Writing GitHub Actions YAML workflows
YAML workflows execute sccripts upon a push or pull request to a repository's branch. There is one GitHub Actions YAML workflow under .github/workflows/regresstion-test.yaml that
sets up the environment to run the tests (columbia-sld's ubuntu-small Docker image) and installs all the open-source linters. Then, it executes code formatting and testing scripts.
Installing open-source tools
vhdl-style-guide
clang-format-10
autopep8
verible
For more information on writing YAML workflows that are compatible with GitHub Actions, and documentation on these open-source linters check out the resources below.
Resources
Disclaimer
Surely, when these scripts and workflows get integrated into the ESP repository they will need to be modified to match the expectations and demands of the team. There is also lots of room for improvement and adding more advanced linting and testing features! Please contact Marian Abuhazi at [email protected] if you have more questions or want to brainstorm more! Happy to help 🤗