Shell - Processing files in loop-CodePudding

I am working on a shell script and got stuck to a point where I need to pass arguments to a function. But it is not currently working as expected. I request you to kindly go through the script and let me know what I am missing here.

If I write below code in the script file (test.sh) directly without any function then it is working fine.

source ./logger.sh
now=$(date  "%Y.%m.%d_%H.%M.%S")
logFile="/path/to/the/directory/${now}.log"
touch $logFile
scriptName="test.sh"
source="/path/to/the/directory"
older=7

while IFS= read -r -d $'\0'; do
  printf "\nIn while: Currently processing file: ${REPLY}\n"
  file=$REPLY
  log 1 $scriptName "main" "Currently processing file: ${file}" $logFile
done < <(find ${source} -maxdepth 1 -type f -mtime  ${older} ! -name '*.FLAG' ! -name '*.FAIL' -print0)

Output in the file

2021-Oct-24 09:37:12            test.sh:    main:   Currently processing file: /path/to/the/directory/file1.txt
2021-Oct-24 09:37:12            test.sh:    main:   Currently processing file: /path/to/the/directory/file2.txt
..... more files ...

However, when I encasulate the script into a function call then it does not work fine.

function moveFiles() {
  source=$1
  older=$2
  logFile=$3

  log 1 $scriptName "moveFiles" "Finding files from ${source} which changed before ${older} days." $logFile

  while IFS= read -r -d $'\0'; do
    printf "\nIn while: Currently processing file: ${REPLY}\n"
    file=$REPLY
    log 1 $scriptName "moveFiles" "Currently processing file: ${file}" $logFile
  done < <(find ${source} -maxdepth 1 -type f -mtime  ${older} ! -name '*.FLAG' ! -name '*.FAIL' -print0)

  log 1 $scriptName "moveFiles" "For loop completed" $logFile

}

source ./logger.sh
now=$(date  "%Y.%m.%d_%H.%M.%S")
logFile="/path/to/the/directory/${now}.log"
touch $logFile
scriptName="test.sh"

moveFiles "/path/to/the/directory" 7 $logFile
log 2 $scriptName "main" "Move files - completed" $logFile

Output in the file

2021-Oct-24 09:43:23            test.sh:    moveFiles:  Finding files from /path/to/the/directory which changed before 7 days.
2021-Oct-24 09:43:23            test.sh:    moveFiles:  For loop completed

The output I am getting when I run the script directly without function, I need to have the same output via function as well. Please help me figuring out the issue.

Also I would like to prefer looping an array of files instead of providing find command output to while loop. I could not find any valid solution, that is why I opted using this approach. If there is any better approach, I would definitely like to opt that.

Note: The log function is in the logger.sh which I included via source ./logger.sh and it seems to be working fine.

EDIT 1: I am looking for a way to store the find command output to array and then send the array to another function and use loop there to process the elements of the array. Is there a better way that I can do? I tried below but it is not working. I always get only 1 element in the array.

list=$(find ${source} -maxdepth 1 -type f -mtime  ${older} ! -name '*.FLAG' ! -name '*.FAIL' -print0)
processInLoop ${list[@]}

In the other function

function processInLoop(){
fileList=("${@}")

}

Thank you in advance.

CodePudding user response：

Tried below with your directory and filters in find statement .

function moveFiles() {
source=$1
older=$2
logFile=$3

echo 1 $scriptName "moveFiles" "Finding files from ${source} which changed before ${older} days." $logFile

find ${source} -maxdepth 1 -type f -mtime  1  -name '*' -print0 |
while IFS= read -r -d ''; do
  echo "\nIn while: Currently processing file: ${REPLY}\n"
  file=$REPLY
  echo $scriptName "moveFiles" "Currently processing file: ${file}" 
  $logFile
done

echo $scriptName "moveFiles" "For loop completed" $logFile

}

now=$(date  "%Y.%m.%d_%H.%M.%S")
logFile="."
touch $logFile
scriptName="test.sh"

moveFiles "." 7 $logFile
echo $scriptName "main" "Move files - completed" $logFile

CodePudding user response：

I am looking for a way to store the find command output to array ...

For that use readarray.

readarray -d '' -t arr < <(find ....)
yourfunction "${arr[@]}"

There are many problems with the script - unquoted variable expansions, using normal variable as array, trying to store zero byte in a string. Check your script with shellcheck.

Quote Variable expansions to prevent word splitting and filename expansion. Don't touch $logFile, do touch "$logFile".

$'\0' is the same as ''. Every string ends with zero byte, two zero bytes is the same as one zero byte, as strings are interpreter up to the first zero byte.

Subjective: I am not a fan of pascal case, I prefer snake case.

list=$(... -print0) - because strngs store up to a zero byte, you can't store zero byte in a variable. You can pipe it via xxd -p to convert to hex, or read it to array as presented above. You should get a warning from bash about it.

list=$(...) ; func ${list[@]} - variable expansion is not quoted, so it undergoes word splitting and filename expansion. list is not an array, so it has only one element. list=$( ... ) assigns a variable, list=( ... ) would assign an array.

Subjective: find ... ! ... - while in history expansion is disabled in non-interactive scripts, I would advise to quote ! anyway just so when you copy stuff to terminal to "test it out", histoty expansion wouldn't be triggered. I would do find ... '!' ...