I have sub-folders(data1,data2,data3) inside a folder(Data) and the python(.py) programs are present inside all the sub-folders with same name. I want to run these python programme in each folder.

Data
    data1/a.py
    data1/b.py
    data1/c.py
    
    data2/a.py
    data2/b.py
    data2/c.py
    
    data3/a.py
    data3/b.py
    data3/c.py

I tried the code below:

 for dir in Data/*
 do
 cd $dir
 python *.py
 done

But it doesn't execute the code inside the sub-folder,can anybody suggest a proper solution for the same.

CodePudding user response：

I don't think you can run multiple Python scripts with one command. But you can use a second for loop to achieve this.

Another issue is that you need to cd back up to the parent directory before the next loop iteration starts; pushd and popd are a nice solution to this.

for dir in Data/*
do
  pushd $dir
  for script in *.py
  do
    python $script
  done
  popd
done

CodePudding user response：

I don't use python so fwiw ...

Assumptions:

• python can work with a directory/path being part of the input
• we're sitting in the parent directory of Data
• we want to execute all *.py scripts located (somewhere) under Data

One idea to streamline the operation:

while read -r script
do
    python "${script}"
done < <(find Data -type f -name "*.py")

Alternatively, limiting ourselves to specific subdirectories:

for script in Data/data{1..3}/*.py
do
    python "${script}"
done

CodePudding user response：

Don't do this with loops. Assuming you want to run all python scripts in and below Data/, and you want those scripts executed with the current working directory equal to the directory in which the script is found, use:

find Data -name '*.py' -execdir ./{} \;

If you don't want to execute all scripts, filter the find appropriately.

For example, if you just want to run scripts name a.py, b.py, or c.py, you might use:

find Data -name '[abc].py' -execdir ./{} \;

If you have poorly named directories, you might also want to use -type f to avoid attempting to execute a directory named a.py. eg find Data -type f -name '[abc].py' -execdir ./{} \;

You can trivially limit the scope by specifying directories: find Data/data1 Data/data3 ... or find Data/data[12] ... or find Data/data[13] ... all work as expected.

CodePudding user response：

Another approach is subshells.

You are cd-ing into the first directory, and then probably getting an error on the next because it doesn't exist. i.e., the first iteration you cd data1/ and none of the others are subdirectories of that.

Assume your structure is under /tmp/. you are logged to /tmp/Data and run your script. The first loop executes cd data1/, so you are now logged to /tmp/Data/data1/. Let's skip the execution for now...

The next iteration issues cd data2/, which does not exist as a subdirectory of /tmp/Data/data1, so it fails, and at that point you should hope that python *.py will fail, or is at least harmlessly idempotent...

So let's try a simple fix. Assuming you are logged to the directory whre the Data subdirectory exists,

for this in Data/*/*.py; do # iterate over the *scripts*
  ( cd "${this%/*}/" && python "$this" )
done

When the subshell exits, the parent is still logged to the directory where Data is. Only the subshell did the cd.

Or if the scripts don't require you be logged to that directory,

for this in Data/*/*.py; do python "$this"; done

That work for you?