I have one log file of mongodb. I want to display all output of grep command in greater than or less than given value"protocol:op_msg 523ms"

sed -n '/2022-09-15T12:26/,/2022-09-15T14:03/p' mongod.log| grep "op_msg 523ms"

output of log file :

7391:2022-11-22T09:23:23.047-0500 I  COMMAND  [conn26] command test.test appName: "MongoDB Shell" command: find { find: "test", filter: { creationDate: new Date(1663252936409) }, lsid: { id: UUID("7c1bb40c-5e99-4281-9351-893e3d23261d") }, $clusterTime: { clusterTime: Timestamp(1669126970, 1), signature: { hash: BinData(0, B141BFD0978167F8C023DFB4AB32BBB117B3CD80), keyId: 7136078726260850692 } }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:337738 cursorExhausted:1 numYields:2640 nreturned:1 queryHash:6F9DC23E planCacheKey:6F9DC23E reslen:304 locks:{ ReplicationStateTransition: { acquireCount: { w: 2641 } }, Global: { acquireCount: { r: 2641 } }, Database: { acquireCount: { r: 2641 } }, Collection: { acquireCount: { r: 2641 } }, Mutex: { acquireCount: { r: 1 } } } storage:{ data: { bytesRead: 28615999, timeReadingMicros: 288402 } } protocol:op_msg 523ms

I have tried below command , but this command is only giving exact value. I need to find all query of log file which is greater than 100ms.

sed -n '/2022-09-15T12:26/,/2022-09-15T14:03/p' mongod.log| grep "op_msg 523ms"

CodePudding user response：

Option 1) You can use grep bash onliner like this to filter mongo queries in specific execution time range:

grep -P '\d ms' /log/mongodb/mongos.log  | while read LINE; do querytime="$(echo "$LINE" | grep -oP '\d ms' | grep -oP '\d ')"; if [ "$querytime" -gt 6000 ]&&[ "$querytime" -lt 7000 ]; then echo "$LINE"; fi; done

Explained:

1. Filter all log lines having Number "ms" at the end.
2. Loop over the queries to extract execution time in querytime variable
3. Check if $querytime between 6000ms and 7000ms
4. If $queritime in the specified range print the current $LINE

Option 2) You can use the --slow MS, --fast MS options from mlogfilter from the mtools package where you can do something like:

   mlogfilter mongod.log --slow 7000 --fast 6000

CodePudding user response：

If all of the lines of interest have protocol:op_msg as the penultimate column, this becomes pretty trivial:

awk '$(NF-1) ~ /protocol:op_msg/ && $NF > 100 { print $NF }'

Note that this completely ignores the units when doing the comparison and will also print lines in which the final column is 250h or 250ns, but that's easy enough to filter for. With a more accurate problem description, a more precise solution is certainly available.