need to create a new field status_code with value-successful by using ingest pipeline when status inside message field has 200 and when status inside message field is 502,404,402 it muse create status_code with value failed.

Figured out how to create the fields but was unable to pick the exact value of status from the logs. Sample logs:

{action:show,count:208,duration:6.38ms,status:200}

How do I write a grok pattern to pick status value alone from these logs? 200 has different logs compared to 404. Hence unable to define a common pattern(each 404 has different log structure as well)

CodePudding user response：

The only way I found to resolve this issue is to write a grok pattern for different log formats. The set processor in ingest node pipeline could be used to add a new field and in the condition of the set processor, I wrote a condtion to check if the status has value 200 or 404 then added a new field called app_statuscode.

Hence if it has 200 status in the message app_statuscode will be success and if it's 404 or 502 or 403 it will be having value failed.

Grok pattern used : %{GREEDYDATA:temp1},status:%{NUMBER:statucode}%{GREEDYDATA:temp2}

This is a sample one I had used it scrapes off value present in status.Need to make the pattern more specific and do not use Greedy data as it picks anything present in the log.

ctx['app_statuscode']=="200" 

Used this condition in set processor to check if status has value 200 or not.

There will be a need to have 2 set processors one for successful one other for failed status code.

ctx['app_statuscode']=="404"||ctx['app_statuscode']=="502"