Parallelize a bash script and wait for each loop to finish-CodePudding

I'm trying to write a script, that we call pippo.R. pippo.R aim, is to run another script (for.sh) in a for loop with a parallelization using two values : nPerm= total number of times the script has to be run permAtTime= number of script that can run at the same time. A very important thing to do, is to wait for each loop to be concluded, thats why I added a file in which all the PID are stored and then I use the wait function to wait for each of them. The main problem of this script is the following error :

./wait.sh: line 2: wait: pid 836844 is not a child of this shell

For reproducibility sake you can put in a folder the following files : pippo.R

nPerm=10
permAtTime=2
cycles=nPerm/permAtTime
for(i in 1:cycles){
  d=1
  system(paste("./for.sh ", i," ",permAtTime,sep=""))
}

for.sh

#!/bin/bash
for X in $(seq $1)
do
 nohup ./script.sh $(($X  ($2 -1)*$1 )) &
 echo $! >> ./save_pid.txt
done
./wait.sh

wait.sh

#!/bin/bash
while read p; do wait $p; done < ./save_pid.txt

Running Rscript pippo.R you will have the explained error. I know that there is the parallel function that can help me in this but for several reasons i cannot use that package. Thanks

CodePudding user response：

You don't need to keep track of PIDs, because if you call wait without any argument, the script will wait for all the child processes to finish.

#!/bin/bash
for X in $(seq $1)
do
 nohup ./script.sh $(($X  ($2 -1)*$1 )) &
done
wait