How to recursively read a folder and its sub-folders and search for a file-CodePudding

I'm new to shell scripting, so I'll do my best to summarize everything.

I am writing a script with the goal of going into a directory, searching for a file (readme.txt) within each subfolder and follow instructions specified in file (readme.txt).

Code I have written so far:

#! /bin/bash

# Creating directory that the files are going to be moved into

if ! (mkdir /dir1/dir2/backupdir> /dev/null); then
    echo "Directory already exists. "

else
    echo "Directory successfully created. "

fi

# Moving old files to backup directory

cd /dir1/dir2/maindir-*
find /dir1 -mindepth 1 -type f -name "readme.txt" | while read z

do
    if grep -i move "$z" > /dev/null; then

        mv /xdir0/$z /dir1/dir2/backupdir
        # /xdir0/ is the parent directory of /xdir1/
        # readme.txt file does not include /xdir0/ so I added it when moving the files
        
    fi
               
done

Current Output: /dir1/dir2/maindir/subfolder1/readme.txt

Within 'maindir' there are several sub-folders.

maindir/subfolder1 maindir/subfolder2 maindir/subfolder3

Each subfolder has a readme.txt file. The contents of readme.txt would look something like this:

Date: 2021-08-12
Applicable Version: xx.x
Description: Quick summary of files

Steps to apply:
 1. move the following files into a backup location
    * xdir1/xdir2/xdir3/xdir4/filename-*.jar
    * xdir1/xdir2/xdir3/xdir4/filename1-*.jar
 2. done. work completed.

The script would parse through each line after "move" is recognized and stop at the "Done. Work completed" line as all files have been moved to the backup directory I created.

Any help would be greatly appreciated.

CodePudding user response：

You can use mkdir -p newfolder which will create it or silently continue if the folder already exists. This way you would not have to check if the folder exists.

You do not need the sleep 1

You can actually first use find -mindepth 1 -type f -name "readme.txt" to find the folders where you have the readme:

find /dir1 -mindepth 1 -type f -name "readme.txt"|while read z
do
  if grep -i move "$z" > /dev/null;then
    ... do your mv magic
  fi
done

CodePudding user response：

The best way is to use a pipe, you start calling find:

# find all 'readme.txt' files from "$DIR" and subdirectories and make a loop
# for each one.
your_backup_location=/home/user/blah/blah/foo/bar

find "$DIR" -name 'readme.txt' -print | while read file
do
    # Assuming this is executed in subdirectory
    # xdir1/xdir2/xdir3/xdir4, where readme.txt was found...
    # $file contains the readme.txt full path from starting dir.
    echo "executing commands from $file" >&2 # print it on stderr
    ( # this will start a subshell, so we can work in the directory where readme.txt is.
        cd "$(dirname "$file")"  # move to dir where readme.txt is located.
        . readme.txt # read commands from readme.txt file.  Protect comment lines with # char.
        # as commands are read into the shell, no need to export the 
        # shell variable ${your_backup_location} (see below)
    ) # exit subshell, so we are back in start directory.
done

and the readme.txt format should be a shell commented text file (explaining in comments what needs to be done) interspersed with shell commands to execute:

# Date: 2021-08-12
# Applicable Version: xx.x
# Description: Quick summary of files

# Steps to apply:
#  1. move the following files into a backup location
#     * xdir1/xdir2/xdir3/xdir4/filename-*.jar
        mv filename-*.jar ${your_backup_location}
#     * xdir1/xdir2/xdir3/xdir4/filename1-*.jar
        mv filename-*.jar ${your_backup_location}
#  2. done. work completed.

This way you will start a subshell on every directory in which you have a readme.txt file. You can specify many filters on find command, so try to output only what is needed to do some actual work, and don't try to spawn a subshell for each subdirectory just to do nothing. And don't use find to just descend one level each time, as you will have a lot of find processes running to do almost nothing.

The idea is not to do unnecessary processing if you have no strict formatting rules to describe the actions to be done. In this case, shell comments allow you to fully describe the operations, while you can better describe the operations with valid shell commands. This saves a lot of grep and sed employed in other answers.