Is good way to show hourly hits count by searching apache/nginx logs in bash?-CodePudding

I want to setup a scheduled daily job in my app server, it will send me a report of hourly hits filtered by the status code. For example, I want to receipt these reports everyday,

#hourly report of 200 status code

200_report.txt

2022-07-18T01:00:00 50
2022-07-18T02:00:00 100
2022-07-18T03:00:00 200
...

40x_reports and 50x_reports give me the similar content while filtered by 40x codes and 50x codes.

I dont want to use any complex tool, it would better I can achieve this by using one or a few bash command.

Any idea?

Thanks

CodePudding user response：

I think rquery is the exact command you are looking for. You will be able to filter and group the content parsed by regex pattern.

[ rquery]$ ./rq -q "parse /\\\"(?P<origip>[^\n]*)\\\" (?P<host>\S ) (\S ) (?P<user>\S ) \[(?P<time>[^\n] )\] \\\"(?P<request>[^\n]*)\\\" (?P<status>[0-9] ) (?P<size>\S ) \\\"(?P<referrer>[^\n]*)\\\" \\\"(?P<agent>[^\n]*)\\\"/|filter status=200 | select truncdate(time,3600), count(1) | group truncdate(time,3600) | sort truncdate(time,3600)" logs/access.log
29/Jun/2022:10:00:00  1000      11
29/Jun/2022:11:00:00  1000      210
29/Jun/2022:12:00:00  1000      43
29/Jun/2022:14:00:00  1000      41
29/Jun/2022:15:00:00  1000      145
[ rquery]$ ./rq -q "parse /\\\"(?P<origip>[^\n]*)\\\" (?P<host>\S ) (\S ) (?P<user>\S ) \[(?P<time>[^\n] )\] \\\"(?P<request>[^\n]*)\\\" (?P<status>[0-9] ) (?P<size>\S ) \\\"(?P<referrer>[^\n]*)\\\" \\\"(?P<agent>[^\n]*)\\\"/|filter status like '50*' | select truncdate(time,3600), count(1) | group truncdate(time,3600) | sort truncdate(time,3600)" logs/access.log
29/Jun/2022:11:00:00  1000      3
29/Jun/2022:12:00:00  1000      2