Bash compgen not parsing wildcard correctly-CodePudding

My goal is to list all files matching foo/** whereas this folder contains files (1 and 2) and subdirectories (bar) with files:

foo/
├── 1
└── bar
    └── 2

Bash has a build-in called compgen. I found out about this function in this answer. Using compgen -G <pattern>, in my case compgen -G 'foo/**' should list all files and folders. However, it only prints out the contents of the first folder (foo):

foo/bar
foo/1

-> foo/bar/2 is missing! I'd expect the following output:

foo/bar
foo/1
foo/bar/2

Is this a bug in bash's compgen or is the glob incorrect?

More experiments

$ compgen -G 'foo/**/*'
foo/bar/2

Using Python3's pathlib.Path.glob:

from pathlib import Path
p=Path(".").resolve()
[print(f) for f in p.glob("foo/**")]

Output:

foo
foo/bar

from pathlib import Path
p=Path(".").resolve()
[print(f) for f in p.glob("foo/**/*")]

Output:

foo/1
foo/bar
foo/bar/2

This version does exactly what I want, however, I'd prefer to minimize dependencies and not use Python at all (bash only).

Bash version 5.0.17(1)-release (x86_64-pc-linux-gnu) on Ubuntu 20.04.
Bash globstar option on/off doesn't change anything
Python 3.6.9

CodePudding user response：

compgen is a tool that exists specifically for generating completions. It does appear to be buggy (insofar as it doesn't honor the globstar option), but it's also not the best tool to use to generate a list of files matching a glob; so there's no real practical reason for that bug to be one anyone cares about.

The best tool is, well, a glob expression.

shopt -s extglob nullglob
files=( foo/** ) # store files matching foo/** in an array
echo "Found ${#files[@]} files:"
printf ' - %q\n' "${files[@]}"

If in your real-world environment you want to get the glob expression to expand from a variable, and only perform globbing and suppress string-splitting, then you'd want to temporarily set IFS to an empty value while performing the files=( $glob ) assignment.