Compare two files using bash script not working-CodePudding

I was trying to compare two files to check the content of one file is in the other or not. Below if the code snippet used, file1 is just numbers and file2 has the details (including the id) for that id in one line. But this snippet is not working, it always says 'not found'. I have read through a lot of SO posts to see what I'm doing wrong; tried adding '-Fxq' flags as well, but to no avail. If I do a manual grep "id" file2.csv, it works. Any idea what is going on? Something like run time bash variable expansion required?

#!/bin/bash
input="file1.csv"
orders="file2.csv"
while IFS= read -r line
do
    echo "$line"
    if grep "$line" "$orders"; then
      echo "Found..."
    else
      echo "NOT FOUND"
    fi
done < "$input"

CodePudding user response：

Suggsting try gawk script:

gawk '
   FNR==NR {file1[NR]=$0; next} # Read all lines file1.csv into array file1
   {print; foundit=match($0,file1[FNR])} # print current line from file2.csv, and test line match corresponding data from file1.csv
   foundit {print "found in line."; next} # when reading file2.csv and mathed. print the match and read next line.
   {print "unmatched. "file1[FNR]} # if got here lines not match. print it.
 ' file1.csv file2.csv

Notice the numbers from file1.csv could be matched as sub-strings in large numbers.

For example if line #1 in file1.csv is 11 and line #1 in file2.csv is 3115 it will match.

It easy to refine the match if you refine the match() function in 2nd line.

CodePudding user response：

In case anyone looking for the solution, The problem was the line read has \r at the end. Once it was cleaned up, grep started matching lines.

#!/bin/bash -x
input="missing.txt"
orders="detailed.csv"
while IFS= read -r line
do
    echo "$line"
    id="${line/$'\r'/}"
    if grep -q "$id" "$orders"; then
      echo "Found..."
    else
      echo "NOT FOUND"
    fi
done < "$input"