How to execute parallel 'for' loop containing if statements-CodePudding

I've this function for processing some images and converting it to pdf. Question is: how to run the for loop in this function in parallel to process the files more quickly?

pdf_compile() {
  start=`date  %s.%N`
  tmpdir=".temp"
  if [ ! -d "$tmpdir" ]; then
    mkdir $tmpdir
  fi
  echo "width: $1px"
  [[ -z $2 ]] && filter="sc1" || filter="$2" 
  echo "filter:" $filter
  count=$(ls | grep .jpg | wc -l)
  dn=1
  for f in *.jpg ; do
    nm="$tmpdir/$1-$filter-$f"
    if [ ! -f "$nm" ]; then
      echo "processing $f ($dn/$count)"
      if [[ "$filter" =~ ^sc1 ]]; then
        convert $f -resize $1x \
        -brightness-contrast 21x33 \
        -enhance -despeckle -sharpen 2 $nm
      elif [[ "$filter" =~ ^sc2 ]]; then
        convert $f -resize $1x \
          -brightness-contrast 19x45 \
          -sharpen 1 -despeckle -despeckle \
          -enhance $nm
      elif [[ "$filter" =~ ^sc3 ]]; then
        convert -resize $1x \
          -brightness-contrast 19x35 \
          -sharpen 1 -enhance -despeckle \
          -gamma 1.05 \
          -brightness-contrast -30x90 \
          -enhance $nm
      fi
      ffmpeg -i $nm $nm -y -hide_banner -loglevel error
    else
      echo "using cached $f ($dn/$count)"
    fi
    ((dn  ))
  done
  n="$(basename $PWD)_$1px.pdf"
  echo "\nconverting to pdf"
  convert $tmpdir/$1-$filter-*.jpg $n
    if [[ "$3" =~ ^y ]]; then
      echo "removing temp"
      rm $tmpdir -r
    fi
  end=`date  %s.%N`
  runtime=$( echo "$end - $start" | bc -l )
  echo "completed in" $runtime "s"
}

I tired this: https://unix.stackexchange.com/a/216475

for f in *.jpg ; do
    ((i=i%N)); ((i  ==0)) && wait
       .......
done

But it didn't improve the processing time

CodePudding user response：

Your code is incomplete and has quite a few issues. Try pasting it into https://shellcheck.net for some guidance.

You could put & at the end of each of the 3 convert statements inside the if statements in the for loop to do them all in the background and then add wait before the final convert that creates the PDF.

for ...
   if ...
      convert ... &
   else ...
      convert ... &
   else ...
      convert ... &
done
...
wait    # for all JPEGs to finish
convert ... XXX.PDF

Rather than create intermediate JPEGs on disk, you could maybe create "Magick Pixel Cache" files by changing the extension to .mpc and see if that is faster. You could also write them to a RAM-based filesystem like /tmp instead of the local, presumably disk-based filesystem.

You give no indication of the image sizes. If you start with multi-thousand pixel wide images and go to several hundred pixels width, you could usefully use "shrink-on-load" feature.

You give little indication of how many pages your PDFs have, nor how long each part of the processing takes. Better specified questions are often easier to answer well...