bash readarray with process substitution on a variable (e.g. in a loop) plus explanation-CodePudding

assuming i have an output/file

1,a,info
2,b,inf
3,c,in

I want to run a while loop with read

while read r ; do 
   echo "$r";
   # extract line to $arr as array separated by ',' 
   # echo the first item of arr
done <<HEREDOC
1,a,info
2,b,inf
3,c,in   
HEREDOC

I specifically want to use readarray and while, but compelling alternatives are welcome too.

There is a specific way to have readarray (mapfile) behave correctly, but i keep forgetting it. this is a Q&A

CodePudding user response：

The solution is readarray -t -d, arr < <(printf "%s," "$r")

The special part is < <(...) because readarray ....
there is no proper reason to be found why it first needs a redirection arrow and then process-substitution.
Neither in tldp process-sub nor SS64 .
My final understanding is that, <(...) opens a named pipe and readarray is waiting for it to close. By moving this in place of a file behind < it is handled by bash as a file input and (anonymously) piped into stdin.

example:

while read r ; do 
   echo "$r";
   readarray -t -d, arr < <(printf "%s," "$r");
   echo "${arr[0]}";
done <<HEREDOC
1,a,info
2,b,inf
3,c,in   
HEREDOC

Anyway this is just a reminder for myself, because i keep forgetting and readarray is the only place where i actually need this.

The question was also answered mostly here, here why the pipe isn't working and somewhat here, but they are difficult to find and the reasoning to comprehend.

for example the shopt -s lastpipe solution is not clear at first, but it turns out that in bash all piped elements are usually not executed in the main shell, thus state changes have no effect on the full program. this command changes the behavior to have the last pipe element execute in main (except in an interactive shell)

shopt -s lastpipe;
while read r ; do 
    echo "$r";       
    printf "%s," "$r"  | readarray -t -d, arr;
    echo "${arr[0]}"; 
    done <<HEREDOC
1,a,info
2,b,inf
3,c,in   
HEREDOC

one alternative to lastpipe would be to do all activity in the sub shell:

while read r ; do 
       echo "$r";
       printf "%s," "$r"  | { 
            readarray -t -d, arr ; 
            echo "${arr[0]}"; 
       }
    done <<HEREDOC
1,a,info
2,b,inf
3,c,in   
HEREDOC

CodePudding user response：

Since compelling alternatives are welcome too and assuming you're just trying to populate arr one line at a time for some reason:

$ cat tst.sh
#!/usr/bin/env bash

while IFS=',' read -a arr ; do
    # extract line to $arr as array separated by ','
    # echo the first item of arr
    echo "${arr[0]}"
done <<HEREDOC
1,a,info
2,b,inf
3,c,in
HEREDOC

$ ./tst.sh
1
2
3

but bear in mind why-is-using-a-shell-loop-to-process-text-considered-bad-practice anyway.