Expand bash array with globbing and extra words-CodePudding

In a CI/CD job, I have a shell variable X defined. It contains one or more words, each of which might have glob operators:

X="foo bar* 'a b c'"

Suppose bar* matches 3 files, bard, bare, and barf. And the file a b c exists and has 2 spaces in it.

I want to create a Bash array Y with these contents:

Y=(--arg foo --arg bard --arg bare --arg barf --arg 'a b c')

In other words: do glob/quote expansion, then map each word w to --arg $w.

A clean solution would allow spaces in the expanded names (to be honest, I'm never going to have that situation in this CI/CD code - but I or someone else might easily copy/paste this technique somewhere else where it does matter).

It would also be awesome if I could do glob/quote expansion without invoking all other possible shell expansions (e.g. process substitution & subshells) - I'm having trouble thinking of how to do that, though.

The only solution I've come up with so far is to use unprotected expansion:

Y=()
for a in $X; do
    Y =(--arg "$a")
done

Is that the best I can do? Is it safe? It works well on the foo and bar* cases, but not well on the a b c case.

CodePudding user response：

You could use declare which has eval capabilities:

#!/bin/bash

x="foo bar* 'a b c'"
declare -a x="($x)"

y=()
for a in "${x[@]}"
do
    y =(--arg "$a")
done

Now it comes down to what you would like x='$HOME \ \$USER $( ls ) to really represent...

CodePudding user response：

To expand globs while honoring quotes (for grouping but not glob-suppression), but not expand variables or handle process substitutions or other dangerous syntax...

X="foo bar* 'a b c'"

IFS=
args=( )
while read -r -d '' word; do
  for item in $word; do
    args =( --arg "$item" )
  done
done < <(xargs printf '%s\0' <<<"$X")

See this running in an online sandbox (where bard, bare and barf exist) at https://replit.com/@CharlesDuffy2/UnlawfulDecentApplicationserver#main.sh

The use of xargs printf '%s\0' has xargs do the work of word-splitting in an almost POSIX-compliant way (if you want something completely POSIX-compliant, you need to use the Python shlex module instead -- other Q&A on the site demonstrates how), and the unquoted expansion with an empty IFS performs globbing only.

CodePudding user response：

Take @Fravadona's answer over this one.

You have to trust that nothing malicious is going to be in that string.

This is a case where you need eval: given

$ touch bar{d,e,f}
$ X="foo bar* 'a b c'"

then this won't get you what you want:

$ y=($X)
$ declare -p y
declare -a y=([0]="foo" [1]="bard" [2]="bare" [3]="barf" [4]="'a" [5]="b" [6]="c'")

however

$ eval "y=($X)"
$ declare -p y
declare -a y=([0]="foo" [1]="bard" [2]="bare" [3]="barf" [4]="a b c")

then

$ Y=(); for elem in "${y[@]}"; do Y =(--arg "$elem"); done
$ declare -p Y
declare -a Y=([0]="--arg" [1]="foo" [2]="--arg" [3]="bard" [4]="--arg" [5]="bare" [6]="--arg" [7]="barf" [8]="--arg" [9]="a b c")

If you have a bad actor, and you get X='nasty $(bad command here)', then you cannot use eval.