I'm somewhat bash challenged and trying to send a large job array through slurm on my institution's cluster. I am way over my limit (which appears to be 1000 jobs per job array) and am having to iteratively parse out the list into blocks of 1000, which is tedious:
sbatch --array=17001-18000 -p <server-name> --time=12:00:00 <my-bash-script>
How might I write a loop to do this? Each job takes about 11 minutes, so I would need to build in a pause in the loop. Otherwise, I suspect SLURM will reject the new batch job. Anyone out there know what to do? Thanks in advance!
Something like this should do what you want
START=1
END=10000
STEP=1000
SLEEP=700 #Just over 11 Minutes (in seconds)
for i in $(seq $START $STEP $END) ; do
JSTART=$i
JEND=$[ $JSTART + $STEP - 1 ]
echo "Submitting with ${JSTART} and ${JEND}"
sbatch --array=${JSTART}-${JEND} -p <server-name> --time=12:00:00 <my-bash-script>
sleep $SLEEP
done
Thank you tomgalpin - this looks like it. I saved this as Job_loop.sh and added the shebang
#!/bin/bashto the first line. Sorry to be slow, but when I run this withsh Job_loop.shI get an error:Job_loop.sh: line 8: JSTART: command not found Job_loop.sh: line 9: JEND: command not found. Thanks in advance!Sorry, ive updated
Thank you very much! Just in case someone else tries this - you just need to switch the order of the sequence to
$START $STEP $ENDand you can subtract 1 fromJENDso that the sequence doesn't overlap.No problem, sorry for the order issue with the seq command, ive updated. Happy to help