Bash: For Loop of Job Arrays - Hoffman2-CodePudding

I am pretty new to bash code, and I have some basic questions.

I have one job array job_array_1.sh, which I am running in Hoffman2.

job_array_1.sh is the following:

#!/bin/bash
#$ -cwd
#$ -o  test.joblog.$JOB_ID.$TASK_ID
#$ -j y
#$ -l h_data=5G,h_rt=00:20:00
#$ -m n
#$ -t 1-5:1

. /u/local/Modules/default/init/modules.sh
module load anaconda3  
#module load python/3.9.6

python3 file1.py $SGE_TASK_ID

If, from the terminal I type qsub job_array_1.sh, this produces 5 different files with names test.joblog.$JOB_ID.$TASK_ID (with the value of t as $TASK_ID). Notice that in this way the 5 jobs start in a parallel way.

I need to create another file call it loop.sh such that it submits the file job_array_1.sh sequentially (in this case twice). So far I have:

#$ -cwd
#$ -j y
#$ -l h_data=3G,h_rt=01:00:00
#$ -m n
for ((i=1; i<=2; i  )); do
    # job submission scripts or shell scripts
    fname_in1="job_array_1.sh"
    ./$fname_in1 &
    wait    
done

When, from the terminal, I type qsub loop.sh this does not produce the 5 files that I have if I do qsub job_array_1.sh. How can I modify the loop.sh file so that the job_array_1.sh produces the 5 files?

CodePudding user response：

I'm guessing wildly here because I don't know anything about your job submission system, but I do know a little about bash and am trying to help. I suspect you need something more like this:

#!/bin/bash
#$ -cwd
#$ -j y
#$ -l h_data=3G,h_rt=01:00:00
#$ -m n
for ((i=1; i<=2; i  )); do
   # job submission scripts or shell scripts
   echo "Loop: $i"
   qsub job_array_1.sh &
done
wait