Text File needs to be read line by line and then used to search a CSV file-CodePudding

Total newbie here.. sorry.

I have a .txt file with IP's (Line by Line):

195.678.333.23
54.221.67.1
32.221.67.2
32.221.67.3
56.221.67.4
32.221.67.5
243.221.67.6
23.221.67.7
34.221.67.8
34.221.67.9

(there about 200 IP's that I need to loop through)

I need to read each line and then search multiple csv files to find these IP's to see if they exist.

the CSV is as such:

3am452525-1;2021-08-25T07:49:03.188353 00:00;195.678.333.23;234.45.78.232;\"pass rscore=16\";[email protected] resolve=ok [email protected] routes= notroutes=alfgdfgd,pr_sdt_Spsdgsgdhidfhgd;sxdgdsgs.hotmail.com;ECDHE-RSA-AES256-GCM-SHA384;NOT;[email protected];ENV_RCPT:dgsgsgsg@{hotmial.com,ENV_RCPT_VERIFY:1;ATTACH_NAME:sdgdsgsgsof.pdf,ATTACH_MIME:application/pdf,ATTACH_TYPE:pdf,ATTACH_OEXT:pdf,ATTACH_CORRUPT:0,ATTACH_PROTECT:0,ATTACH_SIZE:20099;;;;;;sdgsg_fhdh_dfhdh;asrsar_asrar;0;0;0;0;0;0;0;df;8.12.0-2107140000;main-2108250046;;;;sdgsg;;;;none;none;rule=sdgsgs action=ssdgsg-sgddsgr Subject default="sdgs $reyey";rule=odfhddh action=add-header dfhdhdReason="$SpamReason";;;;;;SEND_QID:4363636363,SEND_PROFILE:mail,SEND_RCPTS:dhdfhh;1;1;default_uioyuouyo_RcptInOrg;30677;uMwaF_x-00m7xOj-rho8njCPLr_QKYh3;<B8967386-0450-4709-A382-66D68CE4F3A6@rtce-efax1>;"zdgddadyhadyhad";0.283;1;0.856;Fax-Tdhfdm@urydyd;sdhdfhdhdhfd

The IP exists in $3 field of the CSV file, but if I find the IP then I want the entire line as the output.

Can anyone please help?

I have tired things like:

cat /tmp/IPs_filename.txt | while read line; do grep ${line} csvlog.436334.csv; done

But not helping.

Thanks

CodePudding user response：

awk seems simplest to me.

awk -F';' 'NR==FNR{lst[$0]} $3 in lst{print}' IPs.txt csv

CodePudding user response：

grep -f <txt_file_with_ip_addrs> csv_file1 csv_file2 ...

From grep's man page:

-f, --file=FILE take PATTERNS from FILE

Since the IP addresses contain periods (interpreted as "any character" in regular expressions), you probably also want to use

-F, --fixed-strings PATTERNS are strings

so that the IP addresses are interpreted as literal strings rather than regex.

CodePudding user response：

In bash, without loops: the key here is to use the -f option for grep, and the process substitution to formulate the patterns.

# read the ips into an array
mapfile -t ips < IPs.txt

# grep to find lines with ip in 3rd semicolon-separated field
grep  -E -f <(printf '^[^;] ;[^;] ;%s;\n' "${ips[@]}") CSV.csv