Delete empty files - Improve performance of logic-CodePudding

I am i need to find & remove empty files. The definition of empty files in my use case is a file which has zero lines.

I did try testing the file to see if it's empty However, this behaves strangely as in even though the file is empty it doesn't detect it so.

Hence, the best thing I could write up is the below script which i way too slow given it has to test several hundred thousand files

#!/bin/bash

LOOKUP_DIR="/path/to/source/directory"

cd ${LOOKUP_DIR} || { echo "cd failed"; exit 0; }

for fname in $(realpath */*)
do
        if [[ $(wc -l "${fname}" | awk '{print $1}') -eq 0 ]]
        then
                echo "${fname}" is empty
                rm -f "${fname}"
        fi
done

Is there a better way to do what I'm after or alternatively, can the above logic be re-written in a way that brings better performance please?

CodePudding user response：

Your script is slow beacuse wc reads every file to the end, which is not needed for your purpose. This might be what you're looking for:

#!/bin/bash

lookup_dir='/path/to/source/directory'

cd "$lookup_dir" || exit
for file in *; do
    if [[ -f "$file" && -r "$file" && ! -L "$file" ]]; then
        read < "$file" || echo rm -f -- "$file"
    fi
done

Drop the echo after making sure it works as intended.

Another version, calling the rm only once, could be:

#!/bin/bash

lookup_dir='/path/to/source/directory'

cd "$lookup_dir" || exit
for file in *; do
    if [[ -f "$file" && -r "$file" && ! -L "$file" ]]; then
        read < "$file" || files_to_be_deleted =("$file")
    fi
done
rm -f -- "${files_to_be_deleted[@]}"

CodePudding user response：

To check a "$fname" is a file and is empty or not, use [ -s "$fname" ]:

#!/usr/bin/env sh

LOOKUP_DIR="/path/to/source/directory"

for fname in "$LOOKUP_DIR"*/*; do
  if ! [ -s "$fname" ]; then
    echo "${fname}" is empty
    # remove echo when output is what you want
    echo rm -f "${fname}"
  fi
done

See: help test:

File operators:

...

-s FILE True if file exists and is not empty.