How to track completion and errors of multiple parallel shell commands?-CodePudding

How can I track the completion state and store errors (well, the entire log output) of multiple shell commands running in parallel?

I want to run 3 programs in parallel but I don't want to see their console logs while they are running. I want to instead display a list of all the programs and their exit status UPDATING IN REAL TIME.

# Script is started, no programs have completed
prog1 - Running...
prog2 - Running...
prog3 - Running...

# After some time passes, some programs will have completed
prog1 - Running...
prog2 - FAIL # exit code not 0
prog3 - PASS # exit code 0

Then, after all programs have finished running, I want to display the console logs of any failed program.

prog1 - FAIL
prog2 - FAIL 
prog3 - PASS
--- DONE ---
prog1 encountered an error. LOG:
# Then the entire prog1 log is printed. You would repeat this for every failed program

I know I can run programs in parallel with &

prog1 &
prog2 &
prog3

And I know I can send the stdout of a program to a temp file or something

prog1 > tempfile.txt

But I can't figure out how to track the process exit codes in realtime. I think I can do something with wait and then display only program logs that failed but I don't know how to store the completion status as a variable. I know $? gives you the exit code of the program that was LAST RUN but I want all of them, not just the last run. I need to be able to loop over all the result and then only print the temp logs of the failed programs. Any ideas?

CodePudding user response：

A good & sure point to start with is to read GNU parallel man-pages.

PARALLEL(1)                        parallel                        PARALLEL(1)

NAME
       parallel - build and execute shell command lines from standard input in
       parallel

SYNOPSIS
       parallel [options] [command [arguments]] < list_of_arguments

       parallel [options] [command [arguments]] ( ::: arguments | ::: 
       arguments | :::: argfile(s) | ::::  argfile(s) ) ...

       parallel --semaphore [options] command

       #!/usr/bin/parallel --shebang [options] [command [arguments]]

DESCRIPTION
       GNU parallel is a shell tool for executing jobs in parallel using one
       or more computers. A job can be a single command or a small script that
       has to be run for each of the lines in the input. The typical input is
       a list of files, a list of hosts, a list of users, a list of URLs, or a
       list of tables. A job can also be a command that reads from a pipe. GNU
       parallel can then split the input into blocks and pipe a block into
       each command in parallel.

       If you use xargs and tee today you will find GNU parallel very easy to
       use as GNU parallel is written to have the same options as xargs. If
       you write loops in shell, you will find GNU parallel may be able to
       replace most of the loops and make them run faster by running several
       jobs in parallel.

       GNU parallel makes sure output from the commands is the same output as
       you would get had you run the commands sequentially. This makes it
       possible to use output from GNU parallel as input for other programs.

       For each line of input GNU parallel will execute command with the line
       as arguments. If no command is given, the line of input is executed.
       Several lines will be run in parallel. GNU parallel can often be used
       as a substitute for xargs or cat | bash.

I dare to claim there is hardly a feature that has not been already implemented in the GNU parallel. In a corner-case of indeed being so, you may enjoy the fact of Ole Tange being a StackOverflow member too. So do not hesitate to prototype all your high-level needs using the GNU parallel & if finding some very special piece not available via settings, still may go to the root source of the know-how.

   Reader's guide
       Start by watching the intro videos for a quick introduction:
       http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

       Then look at the EXAMPLEs after the list of OPTIONS. That will give you
       an idea of what GNU parallel is capable of.

       Then spend an hour walking through the tutorial (man
       parallel_tutorial). Your command line will love you for it.

       Finally you may want to look at the rest of this manual if you have
       special needs not already covered.

CodePudding user response：

I know $? gives you the exit code of the program that was LAST RUN but I want all of them, not just the last run. I need to be able to loop over all the result and then only print the temp logs of the failed programs. Any ideas?

I may missunderstood you, but if the reason that you want to see the exit codes of all commands is to figure out if the programm failed, you could utilize set -e .

By adding it at the start of the shell script, it will force the script to exit at the first erroneous command. More info https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html

This way, you will know by the exit code of the script if it failed or not.