Skip to content

Commit b9c0d21

Browse files
QuantumMisakaQuantumMisaka
authored
Cherry-pick Toolchain 202601 to LTS branch (#7321)
* Toolchain: backport 202503 baseline to LTS Partial cherry-pick from 388226b. Excludes forbidden NEP/libtorch installer changes and keeps changes scoped to toolchain. * Backport: [Toolchain] Fix AOCL linking problem and other ver-202503 patch (#6623) Partial cherry-pick from e43b1cd. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: [develop][Toolchain] fix elpa-gpu installation problem in toolchain (#6632) Partial cherry-pick from 3e89604. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Update Cereal URL in FindCereal.cmake (#6722) Partial cherry-pick from b809520. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Fix: Update LibRI URL in FindLibRI.cmake to use codeload for fetching (#6723) Partial cherry-pick from 19a9e54. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Toolchain: Use fixed commit hash instead of master branch (#7193) Partial cherry-pick from d26a604. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Toolchain: Remove redundant exporting variables (#7230) Partial cherry-pick from d54cc6f. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: refactor: find scalapack automatically (#7235) Partial cherry-pick from 60ea82d. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Add CI/CD workflow for Toolchain (#7244) Partial cherry-pick from c2b3b24. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Toolchain: Enable float FFTW linking for building with MKL (#7245) Partial cherry-pick from e6a377d. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Toolchain: Packages updating (#7285) Partial cherry-pick from 0f9d7d9. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Toolchain: Remove notice of downloading master branch (#7303) Partial cherry-pick from e2bd830. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Toolchain 202601 (#7310) Partial cherry-pick from 61734d8. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: Toolchain: Fix package installing issue (#7315) Partial cherry-pick from b9669d3. Scoped to .github, toolchain, and cmake; forbidden NEP/libtorch installer changes and out-of-scope paths are excluded. * Backport: align toolchain CMake modules and workflows Aligns FindMKL/FindFFTW3 and Toolchain workflow files with real-develop final state for the 202503 to 202601 backport. Source commits include 443f89e, f671ce1, 577ae5f, 302ab21, and 61734d8. * Backport: document GCC MKL toolchain entry Completes the 202601 toolchain README update while leaving cuSolverMP/NCCL documentation changes out of the LTS backport scope. * Backport: remove toolchain UI trailing whitespace * Toolchain: fix ELPA environment setup for ABACUS build Set elpa_include in the install path and ensure the include/elpa compatibility symlink is created even when an existing ELPA install is reused. This lets build_abacus_gnu.sh find ELPA after sourcing toolchain/install/setup. * Backport: refine toolchain setup filtering Backports the setup filtering direction from #7319 with an LTS-specific safer filter that preserves conditional blocks. Also keeps the ELPA CPU/NVIDIA install layout discoverable for ABACUS builds and brings in the minimal MPI CXX component change from #7139. * Fix toolchain setup filtering for empty conditionals filter_setup now drops conditional blocks whose contents are fully filtered, preventing generated install/setup files from containing empty if/fi blocks such as the OpenMPI mpi_f08 check. Conditional blocks with runtime path setup, such as ELPA CPU/NVIDIA paths, are preserved. * Add Toolchain smoke test workflow Adds a PR-triggered GNU smoke workflow that performs a real toolchain_gnu.sh installation, validates install/setup, builds ABACUS through build_abacus_gnu.sh, and uploads the relevant logs. * Clean LTS build helper examples Removes develop-only MLALGO, NEP, LibTorch, LibNPY, and DeePMD example options from the toolchain build_abacus helper scripts. These examples were introduced by develop-side changes around PR #7321 but are not supported by the LTS build surface. * Restore LTS DeepKS build examples Restores the LTS build_abacus helper examples after the previous cleanup removed too much. The helpers now keep the LTS DeepKS/LibTorch/LibNPY/DeePMD commented examples while excluding the develop-side NEP and ENABLE_MLALGO examples from PR #7321. * Restore LTS README markers Restores LTS-specific README wording after the toolchain documentation backport: keep the README marked as for LTS and document NEP as not supported while preserving the 202601 version updates and GCC-MKL support documentation. * Fix Toolchain smoke shell and ELPA setup Sets the Toolchain Smoke Test job shell to bash so pipefail is supported in the container. Also aligns ELPA setup generation with PR #7319 by avoiding if blocks in the generated setup file and appending NVIDIA paths only during setup generation when the nvidia install exists. * update toolchain smoke test
1 parent 1dccf29 commit b9c0d21

52 files changed

Lines changed: 994 additions & 1529 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/toolchain_full.yaml‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
name: Toolchain Full Build Test
2+
on:
3+
schedule:
4+
- cron: "0 0 * * 0"
5+
workflow_dispatch:
6+
inputs:
7+
variants:
8+
description: "Comma-separated variants: gnu,intel,cuda"
9+
required: false
10+
default: "gnu,intel,cuda"
11+
jobs:
12+
full-build-gnu:
13+
if: contains(inputs.variants || 'gnu,intel,cuda', 'gnu')
14+
runs-on: X64
15+
container:
16+
image: ghcr.io/deepmodeling/abacus-gnu
17+
steps:
18+
- name: Checkout
19+
uses: actions/checkout@v6
20+
- name: Install requirements
21+
run: |
22+
set -euo pipefail
23+
cd toolchain/root_requirements
24+
sudo ./install_requirements_ubuntu.sh
25+
- name: Build toolchain
26+
run: |
27+
set -euo pipefail
28+
cd toolchain
29+
./toolchain_gnu.sh
30+
- name: Build ABACUS
31+
run: |
32+
set -euo pipefail
33+
cd toolchain
34+
./build_abacus_gnu.sh
35+
cd ..
36+
./bin/abacus --version || true
37+
- name: Upload logs
38+
if: always()
39+
uses: actions/upload-artifact@v7
40+
with:
41+
name: toolchain-full-gnu
42+
path: |
43+
toolchain/compile.log
44+
toolchain/compile.err
45+
toolchain/install/setup
46+
toolchain/build/**/make.log
47+
toolchain/build/**/configure.log
48+
toolchain/build/**/cmake.log
49+
if-no-files-found: ignore
50+
51+
full-build-intel:
52+
if: contains(inputs.variants || 'gnu,intel,cuda', 'intel')
53+
runs-on: X64
54+
container:
55+
image: ghcr.io/deepmodeling/abacus-intel
56+
steps:
57+
- name: Checkout
58+
uses: actions/checkout@v6
59+
- name: Install requirements
60+
run: |
61+
set -euo pipefail
62+
cd toolchain/root_requirements
63+
sudo ./install_requirements_ubuntu.sh
64+
- name: Build toolchain
65+
run: |
66+
set -euo pipefail
67+
cd toolchain
68+
./toolchain_intel.sh
69+
- name: Build ABACUS
70+
run: |
71+
set -euo pipefail
72+
cd toolchain
73+
./build_abacus_intel.sh
74+
cd ..
75+
./bin/abacus --version || true
76+
- name: Upload logs
77+
if: always()
78+
uses: actions/upload-artifact@v7
79+
with:
80+
name: toolchain-full-intel
81+
path: |
82+
toolchain/compile.log
83+
toolchain/compile.err
84+
toolchain/install/setup
85+
toolchain/build/**/make.log
86+
toolchain/build/**/configure.log
87+
toolchain/build/**/cmake.log
88+
if-no-files-found: ignore
89+
90+
full-build-cuda:
91+
if: contains(inputs.variants || 'gnu,intel,cuda', 'cuda')
92+
runs-on: nvidia
93+
steps:
94+
- name: Checkout
95+
uses: actions/checkout@v6
96+
- name: Install requirements
97+
run: |
98+
set -euo pipefail
99+
if [ -f /etc/os-release ]; then . /etc/os-release; fi
100+
if [ "${ID:-}" = "ubuntu" ] || [ "${ID_LIKE:-}" = "debian" ]; then
101+
sudo ./toolchain/root_requirements/install_requirements_ubuntu.sh
102+
elif [ "${ID:-}" = "fedora" ] || [ "${ID_LIKE:-}" = "rhel fedora" ]; then
103+
sudo ./toolchain/root_requirements/install_requirements_fedora.sh
104+
else
105+
exit 1
106+
fi
107+
- name: Prepare CUDA and NVHPC
108+
run: |
109+
set -euo pipefail
110+
if command -v nvidia-smi >/dev/null 2>&1; then
111+
nvidia-smi
112+
cap="$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader | head -n 1 | tr -d '.' | xargs || true)"
113+
fi
114+
if [ -z "${cap:-}" ]; then cap="70"; fi
115+
echo "GPU_CAP=${cap}" >> "$GITHUB_ENV"
116+
if command -v module >/dev/null 2>&1; then
117+
module use /opt/nvidia/hpc_sdk/modulefiles || true
118+
module load nvhpc/26.1 || module load nvhpc-hpcx-cuda12/25.3
119+
else
120+
exit 1
121+
fi
122+
if [ -d /usr/local/cuda ]; then
123+
echo "CUDA_PATH=/usr/local/cuda" >> "$GITHUB_ENV"
124+
fi
125+
- name: Build toolchain
126+
run: |
127+
set -euo pipefail
128+
cd toolchain
129+
./toolchain_gnu.sh --enable-cuda --gpu-ver "${GPU_CAP}"
130+
- name: Build ABACUS
131+
run: |
132+
set -euo pipefail
133+
cd toolchain
134+
./build_abacus_gnu.sh
135+
cd ..
136+
./bin/abacus --version || true
137+
- name: Upload logs
138+
if: always()
139+
uses: actions/upload-artifact@v7
140+
with:
141+
name: toolchain-full-cuda
142+
path: |
143+
toolchain/compile.log
144+
toolchain/compile.err
145+
toolchain/install/setup
146+
if-no-files-found: ignore

‎.github/workflows/toolchain_quick.yaml‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: Toolchain Quick Test
2+
on:
3+
pull_request:
4+
paths:
5+
- toolchain/**
6+
- .github/workflows/toolchain_quick.yaml
7+
push:
8+
branches:
9+
- develop
10+
paths:
11+
- toolchain/**
12+
- .github/workflows/toolchain_quick.yaml
13+
workflow_dispatch:
14+
jobs:
15+
lint-and-sanity:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- name: Checkout
19+
uses: actions/checkout@v6
20+
- name: Install tools
21+
run: |
22+
sudo apt-get update
23+
sudo apt-get install -y dos2unix shellcheck
24+
- name: Shell syntax
25+
run: |
26+
set -euo pipefail
27+
while IFS= read -r -d '' f; do
28+
bash -n "$f"
29+
done < <(find toolchain -type f -name '*.sh' -print0)
30+
- name: Python syntax
31+
run: |
32+
python3 -m compileall toolchain
33+
- name: CRLF check
34+
run: |
35+
set -euo pipefail
36+
if grep -RIl $'\r' toolchain | head -n 1 | grep -q .; then
37+
grep -RIl $'\r' toolchain | head -n 50
38+
exit 1
39+
fi
40+
- name: Shellcheck
41+
run: |
42+
set -euo pipefail
43+
shellcheck --version
44+
find toolchain -type f -name '*.sh' -print0 | xargs -0 -n1 shellcheck -x || true
45+
46+
pack-run-matrix:
47+
runs-on: ubuntu-latest
48+
strategy:
49+
fail-fast: false
50+
matrix:
51+
variant: [gnu, intel, cuda]
52+
steps:
53+
- name: Checkout
54+
uses: actions/checkout@v6
55+
- name: Install tools
56+
run: |
57+
sudo apt-get update
58+
sudo apt-get install -y dos2unix
59+
- name: Pack run
60+
run: |
61+
set -euo pipefail
62+
cd toolchain
63+
rm -rf build install
64+
find . -type f -name '*.sh' -exec chmod +x {} +
65+
if [ "${{ matrix.variant }}" = "gnu" ]; then
66+
./toolchain_gnu.sh --pack-run --skip-system-checks
67+
elif [ "${{ matrix.variant }}" = "intel" ]; then
68+
./toolchain_intel.sh --pack-run --skip-system-checks
69+
elif [ "${{ matrix.variant }}" = "cuda" ]; then
70+
./toolchain_gnu.sh --pack-run --skip-system-checks --enable-cuda --gpu-ver 70
71+
fi
72+
- name: Upload setup
73+
if: always()
74+
uses: actions/upload-artifact@v7
75+
with:
76+
name: toolchain-${{ matrix.variant }}-packrun
77+
path: |
78+
toolchain/install/setup
79+
toolchain/compile.log
80+
toolchain/compile.err
81+
if-no-files-found: ignore

‎.github/workflows/toolchain_smoke.yaml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Toolchain Smoke Test
2+
on:
3+
pull_request:
4+
paths:
5+
- CMakeLists.txt
6+
- cmake/**
7+
- toolchain/**
8+
- .github/workflows/toolchain_*.yaml
9+
workflow_dispatch:
10+
jobs:
11+
smoke-build-gnu:
12+
runs-on: X64
13+
container:
14+
image: ghcr.io/deepmodeling/abacus-gnu
15+
defaults:
16+
run:
17+
shell: bash
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v6
21+
- name: Install requirements
22+
run: |
23+
set -euo pipefail
24+
cd toolchain/root_requirements
25+
sudo ./install_requirements_ubuntu.sh
26+
- name: Build toolchain
27+
run: |
28+
set -euo pipefail
29+
cd toolchain
30+
./toolchain_gnu.sh
31+
- name: Check setup
32+
run: |
33+
set -euo pipefail
34+
cd toolchain
35+
bash -n install/setup
36+
source install/setup
37+
- name: Build ABACUS
38+
run: |
39+
set -euo pipefail
40+
cd toolchain
41+
./build_abacus_gnu.sh
42+
cd ..
43+
. ./toolchain/abacus_env.sh
44+
./bin/abacus --version
45+
- name: Upload logs
46+
if: always()
47+
uses: actions/upload-artifact@v7
48+
with:
49+
name: toolchain-smoke-gnu
50+
path: |
51+
toolchain/compile.log
52+
toolchain/compile.err
53+
toolchain/install/setup
54+
toolchain/build/**/make.log
55+
toolchain/build/**/configure.log
56+
toolchain/build/**/cmake.log
57+
if-no-files-found: ignore

‎CMakeLists.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -247,7 +247,7 @@ if(DEBUG_INFO)
247247
endif()
248248

249249
if(ENABLE_MPI)
250-
find_package(MPI REQUIRED)
250+
find_package(MPI COMPONENTS CXX REQUIRED)
251251
include_directories(${MPI_CXX_INCLUDE_PATH})
252252
target_link_libraries(${ABACUS_BIN_NAME} MPI::MPI_CXX)
253253
add_compile_definitions(__MPI)

‎cmake/FindCereal.cmake‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ if(NOT CEREAL_INCLUDE_DIR)
1515
include(FetchContent)
1616
FetchContent_Declare(
1717
cereal
18-
URL https://github.com/USCiLab/cereal/archive/refs/tags/v1.3.0.tar.gz
18+
URL https://codeload.github.com/USCiLab/cereal/tar.gz/22a1b36
1919
)
2020
FetchContent_Populate(cereal)
2121
set(CEREAL_INCLUDE_DIR ${cereal_SOURCE_DIR}/include)

‎cmake/FindFFTW3.cmake‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,14 @@ if(FFTW3_FOUND)
4949

5050
set(FFTW3_INCLUDE_DIRS ${FFTW3_INCLUDE_DIR})
5151

52+
# Try to extract FFTW version from header
53+
if(FFTW3_INCLUDE_DIR AND EXISTS "${FFTW3_INCLUDE_DIR}/fftw3.h")
54+
file(STRINGS "${FFTW3_INCLUDE_DIR}/fftw3.h" _fftw_ver_line REGEX "^#define[\t ]+FFTW_VERSION[\t ]+\"[^\"]+\"")
55+
if(_fftw_ver_line)
56+
string(REGEX REPLACE "^#define[\t ]+FFTW_VERSION[\t ]+\"([^\"]+)\"" "\\1" FFTW3_VERSION "${_fftw_ver_line}")
57+
endif()
58+
endif()
59+
5260
if(NOT TARGET FFTW3::FFTW3)
5361
add_library(FFTW3::FFTW3 UNKNOWN IMPORTED)
5462
set_target_properties(FFTW3::FFTW3 PROPERTIES

‎cmake/FindLibComm.cmake‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ if(NOT LIBCOMM_DIR)
1616
include(FetchContent)
1717
FetchContent_Declare(
1818
LibComm
19-
URL https://github.com/abacusmodeling/LibComm/archive/refs/tags/v0.1.1.tar.gz
19+
URL https://codeload.github.com/abacusmodeling/LibComm/tar.gz/965bf90
2020
)
2121
FetchContent_Populate(LibComm)
2222
set(LIBCOMM_DIR ${libcomm_SOURCE_DIR})

‎cmake/FindLibRI.cmake‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ if(NOT LIBRI_DIR)
1616
include(FetchContent)
1717
FetchContent_Declare(
1818
LibRI
19-
URL https://github.com/abacusmodeling/LibRI/archive/refs/tags/v0.2.1.1.tar.gz
19+
URL https://codeload.github.com/abacusmodeling/LibRI/tar.gz/e6d78e0
2020
)
2121
FetchContent_Populate(LibRI)
2222
set(LIBRI_DIR ${libri_SOURCE_DIR})

0 commit comments

Comments
 (0)