I want to run multiple dd commands in background, but be able to see the status.

I have the following script.sh:

#!/usr/bin/env bash

for drive in $@
do
  echo "Wiping $drive"
  dd if=/dev/zero of=$drive status=progress &
done
wait
echo "Done."

Which results in the following output:

$ sudo bash ./script.sh /dev/sda /dev/sdb
Wiping /dev/sda
Wiping /dev/sdb
288788992 bytes (289 MB, 275 MiB) copied, 10 s, 28.9 MB/s 14404864 bytes (114 MB, 109 MiB) copied, 4 s, 28.6 MB/s

Is there any way how to output the respective dd statuses below the drive paths? For example:

$ sudo bash ./script.sh /dev/sda /dev/sdb
Wiping /dev/sda
288788992 bytes (289 MB, 275 MiB) copied, 10 s, 28.9 MB/s
Wiping /dev/sdb
14404864 bytes (114 MB, 109 MiB) copied, 4 s, 28.6 MB/s

I tried various redirects, named pipes etc. but wasn't able to achieve such (or similar) output.

CodePudding user response：

Redirect the status of each dd to a file, and print those files repeatedly while clearing the old output.

dd status=progress from GNU coreutils prints its status to stderr, so use 2> to redirect the status info.
The status is updated by overwriting the current line with \r followed by the new status. Since \r only works for single lines, updating four different lines needs terminal control sequences, e.g. ANSI escape codes which can be printed conveniently using commands like clear and tput.

outfile_prefix=/tmp/wipe-status-$$-
for drive in "$@"; do
  (
    echo "Wiping $drive"
    # dummy-version for testing ...
    dd if=/dev/zero of=/dev/null bs=1 count=5M status=progress 2>&1
    # ... if you are happy with the output, replace it with
    # dd if=/dev/zero of="$drive" status=progress 2>&1
  ) > "$outfile_prefix$BASHPID" &
done

clear -x  # clear the currently visible screen
while             # do-while loop
  tput home                  # move cursor to top left
  sed '' "$outfile_prefix"*  # print files like `cat`, but with \n at the end
  jobs %% &> /dev/null       # while jobs are running
do
  sleep 1
done

rm "$outfile_prefix"*

Above code is inefficient if you are wiping many big or slow drives, because the status files keep growing (even though they seem to contain only one line) and we keep printing those growing files over and over again.

If you run into problems, try to increase the sleep time. Maybe sed 's/.*\r//' instead of sed '' can speed things up. tail -c100 would definitively help, but inserts the temp. file names into the output.

The correct way to handle situation would be ...

• Use fifos instead of files (see mkfifo command).
• Remove the status=progress and periodically ...
  • Call kill -s USR1 for all* running dd jobs,
    which causes them to print a single status line.
  • Update some internal status array by reading the fifos*.
  • Print all statuses from the array.

* At some point, only a few dd jobs will run while others already finished and closed their fifos, which complicates that process a bit. This is the main reason I sticked with files for this answer.

CodePudding user response：

Using coprocesses you can do something like this

#! /bin/bash

coproc DD1 { dd if=/dev/zero of=$drive1 status=progress 2>&1; }
coproc DD2 { dd if=/dev/zero of=$drive2 status=progress 2>&1; }
coproc DD3 { dd if=/dev/zero of=$drive3 status=progress 2>&1; }

el=$(tput el)
cuu1=$(tput cuu1)

IFS=
while :
do
    for n in {1..3}
    do
        v="DD$n"
        if read -r -d $'\r' -u ${!v[0]} line
        then
            printf '%s%s: %s\r\n' "$el" "$v" "$line"
        else
            printf '%s: Done\r\n' "$el" "$v"
        fi
    done
    for n in {1..3}
    do
        printf '%s' "$cuu1"
    done
done

my dd does not have status so I'm assuming dd adds a \r instead of \n.