Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pipeline for NVHPC #197

Merged
merged 1 commit into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/bvt-nvhpc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
on:
workflow_call:
inputs:
branch:
type: string
required: false

jobs:
bvt-nvhpc:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.branch }}

- name: install NVHPC 24.9
run: |
curl https://developer.download.nvidia.com/hpc-sdk/ubuntu/DEB-GPG-KEY-NVIDIA-HPC-SDK | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg
echo 'deb [signed-by=/usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg] https://developer.download.nvidia.com/hpc-sdk/ubuntu/amd64 /' | sudo tee /etc/apt/sources.list.d/nvhpc.list
sudo apt-get update -y
sudo apt-get install -y nvhpc-24-9

- name: build and run test with NVHPC 24.9
run: |
PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/24.9/compilers/bin:$PATH; export PATH
cmake -B build-nv -DCMAKE_C_COMPILER=nvc -DCMAKE_CXX_COMPILER=nvc++ -DCMAKE_BUILD_TYPE=Release
cmake --build build-nv -j
ctest --test-dir build-nv -j

- name: run benchmarks
run: |
cd build-nv/benchmarks
./msft_proxy_benchmarks --benchmark_min_warmup_time=0.1 --benchmark_min_time=0.1s --benchmark_repetitions=30 --benchmark_enable_random_interleaving=true --benchmark_report_aggregates_only=true --benchmark_format=json > benchmarking-results.json

- name: archive benchmarking results
uses: actions/upload-artifact@v4
with:
name: benchmarking-results-nvhpc
path: build-nv/benchmarks/benchmarking-results.json
10 changes: 9 additions & 1 deletion .github/workflows/bvt-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,15 @@ jobs:

- name: generate report
run: |
tools/report_generator/build/report_generator tools/report_generator/report-config.json ${{ github.sha }} artifacts benchmarking-report.md
cat <<EOF > benchmarking-report.md
## Benchmarking Report

- Generated for: [Microsoft "Proxy" library](https://github.com/microsoft/proxy)
- Commit ID: [${{ github.sha }}](https://github.com/microsoft/proxy/commit/${{ github.sha }})
- Generated at: $(date -u +"%Y-%m-%dT%H:%M:%SZ")

EOF
tools/report_generator/build/report_generator tools/report_generator/report-config.json

- name: archive benchmarking report
uses: actions/upload-artifact@v4
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/pipeline-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,11 @@ jobs:
uses: ./.github/workflows/bvt-appleclang.yml
name: Run BVT with AppleClang

run-bvt-nvhpc:
uses: ./.github/workflows/bvt-nvhpc.yml
name: Run BVT with NVHPC

report:
uses: ./.github/workflows/bvt-report.yml
name: Generate report
needs: [run-bvt-gcc, run-bvt-clang, run-bvt-msvc, run-bvt-appleclang]
needs: [run-bvt-gcc, run-bvt-clang, run-bvt-msvc, run-bvt-appleclang, run-bvt-nvhpc]
11 changes: 9 additions & 2 deletions .github/workflows/pipeline-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,21 @@ jobs:
with:
branch: release/${{ github.event.inputs.version }}

run-bvt-nvhpc:
needs: prepare-release
name: Run BVT with NVHPC
uses: ./.github/workflows/bvt-nvhpc.yml
with:
branch: release/${{ github.event.inputs.version }}

report:
uses: ./.github/workflows/bvt-report.yml
name: Generate report
needs: [run-bvt-gcc, run-bvt-clang, run-bvt-msvc, run-bvt-appleclang]
needs: [run-bvt-gcc, run-bvt-clang, run-bvt-msvc, run-bvt-appleclang, run-bvt-nvhpc]

draft-release:
runs-on: windows-latest
needs: [run-bvt-gcc, run-bvt-clang, run-bvt-msvc, run-bvt-appleclang]
needs: report
steps:
- uses: actions/checkout@v3
with:
Expand Down
14 changes: 8 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,11 +164,12 @@ The "Proxy" library is a self-contained solution for runtime polymorphism in C++

## <a name="compiler-req">Minimum Requirements for Compilers</a>

| Family | Minimum version | Required flags |
| ------ | --------------- | -------------- |
| clang | 15.0.0 | -std=c++20 |
| gcc | 11.2 | -std=c++20 |
| MSVC | 19.30 | /std:c++20 |
| Family | Minimum version | Required flags |
| ---------- | --------------- | -------------- |
| GCC | 11.2 | -std=c++20 |
| Clang | 15.0.0 | -std=c++20 |
| MSVC | 19.30 | /std:c++20 |
| NVIDIA HPC | 24.1 | -std=c++20 |

## Build and Run Tests with CMake

Expand All @@ -182,9 +183,10 @@ ctest --test-dir build -j

## Related Resources

- November, 2024: [Analyzing the Performance of the “Proxy” Library](https://devblogs.microsoft.com/cppblog/analyzing-the-performance-of-the-proxy-library/)
- September, 2024: [Announcing the Proxy 3 Library for Dynamic Polymorphism](https://devblogs.microsoft.com/cppblog/announcing-the-proxy-3-library-for-dynamic-polymorphism/)
- April, 2024: [Published ISO C++ proposal P3086R2: Proxy: A Pointer-Semantics-Based Polymorphism Library](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p3086r2.pdf)
- August, 2022: [proxy: Runtime Polymorphism Made Easier Than Ever](https://devblogs.microsoft.com/cppblog/proxy-runtime-polymorphism-made-easier-than-ever/)
- September, 2024: [Announcing the Proxy 3 Library for Dynamic Polymorphism](https://devblogs.microsoft.com/cppblog/announcing-the-proxy-3-library-for-dynamic-polymorphism/)

## Contributing

Expand Down
30 changes: 13 additions & 17 deletions tools/report_generator/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
// Licensed under the MIT License.

#include <cstdio>
#include <chrono>
#include <filesystem>
#include <format>
#include <fstream>
Expand All @@ -15,19 +14,19 @@
#include <nlohmann/json.hpp>

struct EnvironmentInfo {
std::string Name;
std::string Description;
std::string Path;

friend void to_json(nlohmann::json& j, const EnvironmentInfo& e) {
j = nlohmann::json{
{"Name", e.Name},
{"Description", e.Description}
{"Description", e.Description},
{"Path", e.Path}
};
}

friend void from_json(const nlohmann::json& j, EnvironmentInfo& e) {
j.at("Name").get_to(e.Name);
j.at("Description").get_to(e.Description);
j.at("Path").get_to(e.Path);
}
};

Expand All @@ -54,13 +53,15 @@ struct MetricInfo {
struct ReportConfig {
std::string TargetName;
double YellowIndicatorThreshold;
std::string OutputPath;
std::vector<EnvironmentInfo> Environments;
std::vector<MetricInfo> Metrics;

friend void to_json(nlohmann::json& j, const ReportConfig& rc) {
j = nlohmann::json{
{"TargetName", rc.TargetName},
{"YellowIndicatorThreshold", rc.YellowIndicatorThreshold},
{"OutputPath", rc.OutputPath},
{"Environments", rc.Environments},
{"Metrics", rc.Metrics}
};
Expand All @@ -69,6 +70,7 @@ struct ReportConfig {
friend void from_json(const nlohmann::json& j, ReportConfig& rc) {
j.at("TargetName").get_to(rc.TargetName);
j.at("YellowIndicatorThreshold").get_to(rc.YellowIndicatorThreshold);
j.at("OutputPath").get_to(rc.OutputPath);
j.at("Environments").get_to(rc.Environments);
j.at("Metrics").get_to(rc.Metrics);
}
Expand Down Expand Up @@ -96,7 +98,7 @@ std::unordered_map<std::string, double> Parse(const std::filesystem::path& file)
return result;
}

void GenerateReport(const std::filesystem::path& config_path, const std::string& commit_id, const std::filesystem::path& source, const std::filesystem::path& output) {
void GenerateReport(const std::filesystem::path& config_path) {
ReportConfig config;
{
nlohmann::json obj;
Expand All @@ -109,17 +111,11 @@ void GenerateReport(const std::filesystem::path& config_path, const std::string&
std::vector<std::unordered_map<std::string, double>> benchmarks;
benchmarks.reserve(config.Environments.size());
for (auto& environment : config.Environments) {
benchmarks.push_back(Parse(source / std::format("benchmarking-results-{}", environment.Name) / "benchmarking-results.json"));
benchmarks.push_back(Parse(environment.Path));
}
std::ofstream out;
out.exceptions(std::ios_base::failbit | std::ios_base::badbit);
out.open(output, std::ios_base::out | std::ios_base::trunc | std::ios_base::binary);
out << "## Benchmarking Report\n";
out << "\n";
out << "- Generated for: [Microsoft \"Proxy\" library](https://github.com/microsoft/proxy)\n";
out << "- Commit ID: [" << commit_id << "](https://github.com/microsoft/proxy/commit/" << commit_id << ")\n";
out << "- Generated at: " << std::format("{:%FT%TZ}", std::chrono::utc_clock::now()) << "\n";
out << "\n";
out.open(config.OutputPath, std::ios_base::out | std::ios_base::app | std::ios_base::binary);
out << "| |";
for (auto& environment : config.Environments) {
out << " " << environment.Description << " |";
Expand Down Expand Up @@ -162,12 +158,12 @@ void GenerateReport(const std::filesystem::path& config_path, const std::string&
}

int main(int argc, char** argv) {
if (argc != 5) {
puts("Usage: report_generator <config file path> <commit ID> <benchmarking results directory> <output file path>");
if (argc != 2) {
puts("Usage: report_generator <config file path>");
return 0;
}
try {
GenerateReport(argv[1], argv[2], argv[3], argv[4]);
GenerateReport(argv[1]);
} catch (const std::exception& e) {
fprintf(stderr, "An error occurred: %s\n", e.what());
return 1;
Expand Down
21 changes: 13 additions & 8 deletions tools/report_generator/report-config.json
Original file line number Diff line number Diff line change
@@ -1,22 +1,27 @@
{
"TargetName": "`proxy`",
"YellowIndicatorThreshold": 5.0,
"OutputPath": "benchmarking-report.md",
"Environments": [
{
"Name": "msvc",
"Description": "MSVC on Windows Server 2022 (x64)"
"Description": "MSVC on Windows Server 2022 (x64)",
"Path": "artifacts/benchmarking-results-msvc/benchmarking-results.json"
},
{
"Name": "gcc",
"Description": "GCC on Ubuntu 24.04 (x64)"
"Description": "GCC on Ubuntu 24.04 (x64)",
"Path": "artifacts/benchmarking-results-gcc/benchmarking-results.json"
},
{
"Name": "clang",
"Description": "Clang on Ubuntu 24.04 (x64)"
"Description": "Clang on Ubuntu 24.04 (x64)",
"Path": "artifacts/benchmarking-results-clang/benchmarking-results.json"
},
{
"Name": "appleclang",
"Description": "Apple Clang on macOS 15 (ARM64)"
"Description": "Apple Clang on macOS 15 (ARM64)",
"Path": "artifacts/benchmarking-results-appleclang/benchmarking-results.json"
},
{
"Description": "Nvidia HPC on Ubuntu 24.04 (x64)",
"Path": "artifacts/benchmarking-results-nvhpc/benchmarking-results.json"
}
],
"Metrics": [
Expand Down