Elasticsearch reindex/update_by_query to replace incorrect timestamp-CodePudding

I've spent quite a bit of time looking for a solution with little success.

I have a number of indices that were indexed using the wrong field as timestamp. I need to replace the current @timestamp field with another text field that contains the correct timestamp

This illustrates what I need to do:

POST indexname-2022.08.28/_update_by_query
{
    "query": {
      "match_all": {}
    },
    "script": {
    "source": "ctx._source['@timestamp'] = ctx._source['Event-Timestamp'])"
    }
}

"@timestamp" is a date field with pattern "Feb 2, 2022 @ 14:54:35.379"

"Event-Timestamp" is a text field with pattern "Feb 2 2022 14:54:34 MST"

It appears that update_by_query is the best option but I'm unable to get syntax correct. I know that "Event-Timestamp" needs to be converted to a date field before it a can be used to replace the "@timestamp". Any help would be appreciated.

Thanks

rlk

CodePudding user response：

Welcome on StackOverflow :D

Here is the documentation about how to use date time in painless script. There are multiple example of parsing date in script.

For your case I created a custom date formatter. Here is an example query working for you demo case:

POST indexname-2022.08.28/_update_by_query
{
  "query": {
    "match_all": {}
  },
  "script": {
    "source": """
        DateTimeFormatter dtf = DateTimeFormatter.ofPattern("MMM d yyyy HH:mm:ss z");
        ctx._source['@timestamp'] = ZonedDateTime.parse(ctx._source['Event-Timestamp'], dtf)
        """
    }
}

If the Event-Timestamp format is consistent it should do the works !

Tell me if you run into troubles with this script.