Easy way to extract IP and referer from nginx log file-CodePudding

How to find ip and referer which contains facebook/instagram/twitter/etc using awk or others possible commands, to get this:

1.2.3.4 https://l.instagram.com/
4.3.2.1 https://facebook.com/

Nginx log file has standart format:

1.2.3.4 - - [02/Oct/2021:06:07:08  0300 - -] 200 "GET /index.php HTTP/2.0" 6620 "https://l.instagram.com/" "Mozilla/5.0 (Linux; Android 9; SM-N950F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Mobile Safari/537.36" "-"

I'm using

awk '{print $1}' access.log

and

awk -F\" '{print $2}' access.log

But I don't know how to get ip and referer together.

CodePudding user response：

awk

$ awk '{gsub(/,/,"");print $1, $13}'

sed

$ sed -E 's/^(.[^ ]*).*"(https.[^"]*).*/\1 \2/'

output

1.2.3.4 https://l.instagram.com/

CodePudding user response：

$ awk -F'[ "]' '{print $1, $16}' file
1.2.3.4 https://l.instagram.com/