I am writing a bash script to count number of files given user parameters of path and exclude path. I construct a command using my bash script, where the last few line looks like this

command="find . $exclude | wc -l"
echo $command

So for example, it might result in

find . 

Or if I set exclude to be '-not \( -path "./special_dir" -prune \)', then the command would be

find .   -not \( -path "./special_directory" -prune \) 

Both commands I can copy and paste into a command line and everything is fine.

The problem comes when I try to run it.

exclude=""
command="find $. $exclude | wc -l"
echo $command
results=$($command)
echo "result=$results"

Above works, but the snippet below would fail.

exclude='-not \( -path "./special_dir" -prune \)'
command="find $. $exclude | wc -l"
echo $command
results=$($command)
echo "result=$results"

returns

find . -not \( -path "./special_dir" -prune \) | wc -l
find: paths must precede expression: \(
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

But if I use eval to run command, even the exclude is fine

exclude='-not \( -path "./special_dir" -prune \)'
command="find $. $exclude | wc -l"
echo $command
results=$(eval $command)
echo "result=$results"

Returns

find .   -not \( -path "./special_dir" -prune \)  | wc -l
result=872 

I don't understand what's wrong with just running the command with exclude directory. My guess is it has to do some with kind of escaping special characters?

CodePudding user response：

what's wrong with just running the command with exclude directory.

Pipeline | is parsed before variables are expanded.

Unquoted variable expansion undergo word splitting and filename expansion. This is irrelevant of quotes and backslashes inside the string - you can put as many backslashes as you want, the result of the expansion still will be split on spaces. https://www.gnu.org/software/bash/manual/bash.html#Shell-Expansions

y guess is it has to do some with kind of escaping special characters?

No, it does not matter what is the content of the variable. It's important how you use the variable.

Above works, but the snippet below would fail.

Yes, \( is an invalid argument for find, it should be (. The result of expansion is not parsed for escape sequences, they are preserved literally, so \( stays to be \(, two characters. In comparison, when the line is parsed, then quoting is parsed. https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation

In either case, you want to be using Bash arrays and want to do a lot of experimenting with quoting and expansion in shell. The following could be enough to get you started:

findargs=( -not \( -path "./special_directory" -prune \) )
find . "${findargs[@]}"

Check your scripts with shellcheck. Do not store commands in variables - use functions and bash arrays. Quote variable expansions. eval is very unsafe, and if the path comes from the user, shouldn't be used. Use printf "%q" to quote a string for re-eval-ing. See https://mywiki.wooledge.org/BashFAQ/050 , https://mywiki.wooledge.org/BashFAQ/048 , and read many introductions to bash arrays.

CodePudding user response：

The reason you typically have to escape the parentheses in a find command is that parentheses have special meaning in bash. Bash sees a command like find . -not ( -path "./special_dir" -prune ) and tries to interpret those parentheses as shell syntax instead of literal characters that you want to pass to your command. It can't do that in this context, so it complains. The simple fix is to escape the parentheses to tell bash to treat them literally.

To reiterate: those backslashes are NOT syntax to the find command, they are there to tell bash not to treat a character specially even if it normally has special meaning. Bash removes those backslashes from the command line before executing the command so the find program never sees any backslashes.

Now in your use case, you're putting the command in a variable. Variable expansion happens after bash has parsed the contents of the command line, so escape characters inside the variable (like '\') won't be treated like escapes. They'll be treated like any normal character. This means your backslashes are being passed as literal characters to your command, and find doesn't actually know what to do with them because they aren't part of that command's syntax.

Further reading on the bash parser

The simplest thing you can do: Just don't include the backslashes. Something like this will work fine:

exclude='-not ( -path ./special_dir -prune )' 
command="find . $exclude"
$command

The parentheses don't actually need to be escaped here because the parsing step that would interpret them as special characters happens before variable expansion, so they'll just be treated literally.

But really, you might be better off using a function. Variables are meant to hold data, functions are meant to hold code. This will save you a lot of headaches trying to understand and work around the parsing rules.

A simple example might be:

normal_find() {
  find .
}

find_with_exclude() {
  find . -not \( -path ./special_dir -prune \)
}

my_command=normal_find
#my_command=find_with_exclude #toggle between these however you like

results=$( $my_command | wc -l )
echo "results=$results"