Elasticsearch, upsert a document with script when the index does not exist-CodePudding

I'm receiving some payloads in a logstash, that I push in Elastic in a monthly rolling index with a script that allows me to override the fields depending on the order of the status of those payloads.

Example :

{
 "id" : "abc",
 "status" : "OPEN",
 "field1" : "foo",
 "opening_ts" : 1234567
}

{
 "id" : "abc",
 "status" : "CLOSED",
 "field1" : "bar",
 "closing_ts": 7654321
}

I want that, even if i receive the payload OPEN after the CLOSE for the id "abc", my elastic document to be :

{
 "_id" : "abc",
 "status": "CLOSED",
 "field1" : "bar",
 "closing_ts": 7654321,
 "opening_ts" : 1234567
}

I order to guarantee that, i have added a script in my elastic output plugin in logstash

script => "
      if (ctx._source['status'] == 'CLOSED') {
        for (key in params.event.keySet()) {
          if (ctx._source[key] == null) {
            ctx._source[key] = params.event[key]
          }
        }
      } else {
        for (key in params.event.keySet()) {
          ctx._source[key] = params.event[key]
        }
      }
      "

Buuuuut, adding this script also added an extra step between the implicit "PUT" on the index, and if the target index does not exist, the script will fail and the whole document will never be created. (Nor the index)

Do you know how could i handle an error in this scripts ?

CodePudding user response：

You need to resort to scripted upsert:

output {
  elasticsearch {
    index => "your-index"
    document_id => "%{id}"
    action => "update"
    scripted_upsert => true
    script => "... your script..."
  }
}