Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci): Balance splits across benchmarking CI jobs according to the number of CPU cores #5099

Merged
merged 4 commits into from
Jan 13, 2025
Merged

fix(ci): Balance splits across benchmarking CI jobs according to the number of CPU cores #5099

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8b32d56
fix: Balance splits across CI jobs according to number of CPU cores p…
ddyurchenko Jan 13, 2025
e4f563d
Fail CI job when GROUP_SIZE is larger than number of CPU cores
ddyurchenko Jan 13, 2025
667e212
Improve variable name
ddyurchenko Jan 13, 2025
1658a3f
Fix the round up formula for group size
ddyurchenko Jan 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 31 additions & 26 deletions benchmark/sirun/runall.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/bin/bash

set -e

# Temporary until merged to master
wget -O sirun.tar.gz https://github.com/DataDog/sirun/releases/download/v0.1.10/sirun-v0.1.10-x86_64-unknown-linux-musl.tar.gz \
&& tar -xzf sirun.tar.gz \
Expand Down Expand Up @@ -34,46 +36,49 @@ echo "using Node.js ${VERSION}"
CPU_AFFINITY="${CPU_START_ID:-24}" # reset for each node.js version
SPLITS=${SPLITS:-1}
GROUP=${GROUP:-1}
BENCH_COUNT=0

BENCH_COUNT=0
for D in *; do
if [ -d "${D}" ]; then
BENCH_COUNT=$(($BENCH_COUNT+1))
cd "${D}"
variants="$(node ../get-variants.js)"
for V in $variants; do BENCH_COUNT=$(($BENCH_COUNT+1)); done
cd ..
fi
done

# over count so that it can be divided by bash as an integer
BENCH_COUNT=$(($BENCH_COUNT+$BENCH_COUNT%$SPLITS))
GROUP_SIZE=$(($BENCH_COUNT/$SPLITS))

run_all_variants () {
local variants="$(node ../get-variants.js)"

node ../squash-affinity.js

for V in $variants; do
echo "running ${1}/${V} in background, pinned to core ${CPU_AFFINITY}..."

export SIRUN_VARIANT=$V

(time node ../run-one-variant.js >> ../results.ndjson && echo "${1}/${V} finished.") &
((CPU_AFFINITY=CPU_AFFINITY+1))
done
}
GROUP_SIZE=$(($(($BENCH_COUNT+$SPLITS-1))/$SPLITS)) # round up

BENCH_INDEX=0
BENCH_END=$(($GROUP_SIZE*$GROUP))
BENCH_START=$(($BENCH_END-$GROUP_SIZE))

if [[ ${GROUP_SIZE} -gt 24 ]]; then
echo "Group size ${GROUP_SIZE} is larger than available number of CPU cores on Benchmarking Platform machines (24 cores)"
exit 1
fi

for D in *; do
if [ -d "${D}" ]; then
if [[ ${BENCH_INDEX} -ge ${BENCH_START} && ${BENCH_INDEX} -lt ${BENCH_END} ]]; then
cd "${D}"
run_all_variants $D
cd ..
fi
cd "${D}"
variants="$(node ../get-variants.js)"

node ../squash-affinity.js

for V in $variants; do
if [[ ${BENCH_INDEX} -ge ${BENCH_START} && ${BENCH_INDEX} -lt ${BENCH_END} ]]; then
echo "running $((BENCH_INDEX+1)) out of ${BENCH_COUNT}, ${D}/${V} in background, pinned to core ${CPU_AFFINITY}..."

export SIRUN_VARIANT=$V

(time node ../run-one-variant.js >> ../results.ndjson && echo "${D}/${V} finished.") &
((CPU_AFFINITY=CPU_AFFINITY+1))
fi

BENCH_INDEX=$(($BENCH_INDEX+1))
done

BENCH_INDEX=$(($BENCH_INDEX+1))
cd ..
fi
done

Expand Down
Loading