Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix path to singularity image and datalad add singularity image #9

Open
wants to merge 4 commits into
base: mit-slurm
Choose a base branch
from
Open

fix path to singularity image and datalad add singularity image #9

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3edd46f
Merge pull request #7 from PennLINC/mit-slurm
smeisler Aug 23, 2021
a0857e7
change srun to sbatch; add tmpdir
smeisler Aug 23, 2021
24deb10
fix fmriprep container path
jsmentch Aug 23, 2021
a9315ad
add datalad get singularity iamge
jsmentch Aug 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 18 additions & 9 deletions scripts/mit_slurm/bootstrap_fmriprep.sh
100755 → 100644
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,18 @@ fi
BIDSINPUT=$1
if [[ -z ${BIDSINPUT} ]]
then
echo "Required argument is an identifier of the BIDS source"
echo "First required argument is an identifier of the BIDS source"
# exit 1
fi

TMPDIR=$2
if [[ -z ${TMPDIR} ]]
then
echo "Second required argument is a path to temporary working directory"
# exit 1
fi
mkdir -p "${TMPDIR}"

# Is it a directory on the filesystem?
BIDS_INPUT_METHOD=clone
if [[ -d "${BIDSINPUT}" ]]
Expand Down Expand Up @@ -94,6 +102,7 @@ fi

CONTAINERDS=///repronim/containers
datalad install -d . --source ${CONTAINERDS}
datalad get containers/images/bids/bids-fmriprep--20.2.3.sing

## the actual compute job specification
cat > code/participant_job.sh << "EOT"
Expand All @@ -111,7 +120,7 @@ pushgitremote="$2"
subid="$3"
# change into the cluster-assigned temp directory. Not done by default in SGE
# TODO: change to local tempdir assigned by SLURM
cd ${CBICA_TMPDIR}
cd ${TMPDIR}
# OR Run it on a shared network drive
# cd /cbica/comp_space/$(basename $HOME)
# Used for the branch names and the temp dir
Expand Down Expand Up @@ -151,7 +160,7 @@ datalad run \
-i code/fmriprep_zip.sh \
-i inputs/data/${subid} \
-i inputs/data/*json \
-i containers/.datalad/environments/fmriprep-20-2-3/image \ # FIX!!
-i containers/images/bids/bids-fmriprep--20.2.3.sing
--explicit \
-o ${subid}_fmriprep-20.2.3.zip \
-o ${subid}_freesurfer-20.2.3.zip \
Expand Down Expand Up @@ -179,9 +188,8 @@ cat > code/fmriprep_zip.sh << "EOT"
set -e -u -x
subid="$1"
mkdir -p ${PWD}/.git/tmp/wdir
# TODO: fix path to singularity image
singularity run --cleanenv -B ${PWD} \
containers/.datalad/environments/fmriprep-20-2-3/image \ # FIX!!
containers/images/bids/bids-fmriprep--20.2.3.sing
inputs/data \
prep \
participant \
Expand All @@ -202,7 +210,8 @@ rm -rf prep .git/tmp/wkdir
EOT

chmod +x code/fmriprep_zip.sh
cp ${FREESURFER_HOME}/license.txt code/license.txt
#cp ${FREESURFER_HOME}/license.txt code/license.txt
cp /om2/user/smeisler/TheWay/scripts/mit_slurm/license.txt code/license.txt #REMOVE THIS LATER

mkdir logs
echo .SLURM_datalad_lock >> .gitignore
Expand All @@ -226,14 +235,14 @@ wget -qO- ${MERGE_POSTSCRIPT} >> code/merge_outputs.sh
# SLURM SETUP START - remove or adjust to your needs
################################################################################
env_flags="--export=DSLOCKFILE=${PWD}/.SLURM_datalad_lock"
echo '#!/bin/bash' > code/srun_calls.sh
echo '#!/bin/bash' > code/sbatch_calls.sh
dssource="${input_store}#$(datalad -f '{infos[dataset][id]}' wtf -S dataset)"
pushgitremote=$(git remote get-url --push output)
eo_args="-e ${PWD}/logs -o ${PWD}/logs"
for subject in ${SUBJECTS}; do
echo "srun ${env_flags} --job-name=fp${subject} ${eo_args} \
echo "sbatch ${env_flags} --job-name=fp${subject} ${eo_args} \
${PWD}/code/participant_job.sh \
${dssource} ${pushgitremote} ${subject} " >> code/srun_calls.sh
${dssource} ${pushgitremote} ${subject} " >> code/sbatch_calls.sh
done
datalad save -m "SLURM submission setup" code/ .gitignore

Expand Down