grokdebugger validates entries of a log that logstash eventually refuses-CodePudding

The

CodePudding user response：

The main responsible of my trouble is the:

grok { match => [

instead of:

grok { match => {

But after that, I had to change:

the timestamp definition to a %{TIMESTAMP_ISO8601:timestamp}

the date match

and in the date match add a target to it to avoid a

to avoid a _dateparsefailure.

@timestamp: Mar 16, 2022 @ 09:14:22.002 @version: 1 class: f.e.service.AbstractSparkDataset host: debian level: INFO logmessage: Un dataset a été sauvegardé dans le fichier parquet /data/tmp/balanceComptesCommunes_2019_2019. thread: http-nio-9090-exec-10 timestamp: 2022-03-16T06:34:09.394Z _id: 8R_KkX8BBIYNTaMw1Jfg _index: ecoemploimetier-2022.03.16 _score: - _type: _doc

I eventually corrected my logstash config file like that:

input { file { path => "/home/[...]/myLog.log" sincedb_path => "/dev/null" start_position => "beginning" codec => multiline { pattern => "^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.*" negate => "true" what => "previous" } } } filter { #If log line contains tab character followed by 'at' then we will tag that entry as stacktrace if [message] =~ "\tat" { grok { match => ["message", "^(\tat)"] add_tag => ["stacktrace"] } } grok { match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} \[(?<thread>(.*?) )\] %{LOGLEVEL:level} %{GREEDYDATA:class} - (?<logmessage>.*)" } } date { # 2022-03-16 07:32:24,860 match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss,SSS" ] target => "timestamp" } # S'il n'y a pas d'erreur de parsing, supprimer le message d'origine, non parsé if "_grokparsefailure" not in [tags] { mutate { remove_field => [ "message", "path" ] } } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["localhost:9200"] index => "ecoemploimetier-%{ YYYY.MM.dd}" } }