Logstash drop logs if field does not have value-CodePudding

I have the following logstash.conf

input {
    tcp {
       port => 5000
      # codec => json { charset=>"UTF-8" }           

    }

    stdin {}
}

filter {

    if "userId" not in [contextMap] {
     drop {}
    }


    json {
            source => "message"
    }

 }

 output {
    elasticsearch {
            hosts => ["localhost:9200"]
            index => "logs"
            user => "user"
            password => "password"
    }

    stdout {}
 }

And I have json request printed in logstash

{
"message" => " testing ,essage",
"instant" => {
    "nanoOfSecond" => 267000000,
    "epochSecond" => 1631632671
},
"thread" => "http-nio-8080-exec-1",
"level" => "INFO",
"loggerFqcn" => "org.apache.logging.slf4j.Log4jLogger",
"loggerName" => "",
"endOfBatch" => false,
"@version" => "1",
"@timestamp" => 2021-09-14T15: 17: 51.504Z,
"threadPriority" => 5,
"hostname" => "hostname",
"contextMap" => {
    "requestId" => "f6f65bab-bdd1-4cca-ba09-ff0d06d16e41",
    "userId" => "bc1289cd-d3ed-43ec-87ba-9c5ff9c5f205",
    "moduleName" => "LOGGER_SERVICE",
},
"host" => "hostname",
"threadId" => 139,
"port" => 55555
}

What I want to achieve is that where the field contextMap does not has field userId, logs should be droped.

Could someone tell me ware I am doing wrong?

CodePudding user response：

if "userId" not in [contextMap] can be used for either a sub-string match if [contextMap] is a string, or an array membership test if [contextMap] is an array. It cannot be used to test if a key exists in a hash. That said, you can use

if ! [contextMap][userId] { drop {} }

CodePudding user response：

I managed to solve the problems. The issue was I didn't want to certain logs to be loaded to to elastic search only with specific field. so I moved the filter to the output section. the complete logstash config file is as below

input {
  tcp {
   port => 5000
  # codec => json { charset=>"UTF-8" }           

  }

  stdin {}
}

filter {
  json {
     source => "message"
  }
}

output {
 if [contextMap][requestId]{
   elasticsearch {
        hosts => ["localhost:9200"]
        index => "logs"
        user => "user"
        password => "password"
   }
 }
 stdout {}
}

Home help someone else