How to convert the particular item in the filebeat message to lowercase using elastic search process-CodePudding

I am simulating the below code in elastic search, how to convert the event.action in the below code from Query to lowercase "query" as expected in the output.

The below simulation done in the elastic devtools console:

POST /_ingest/pipeline/_simulate
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "dissect": {
          "field" : "message",
          "pattern" : "%{@timestamp}\t%{->} %{process.thread.id} %{event.action}\t%{message}"
        },
        "set": {
          "field": "event.category",
          "value": "database"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "message": "2020-10-22T20:28:26.267397Z\t   9 Query\tset session"
      }
    }
  ]
}

Expected output

{
  "docs" : [
    {
      "doc" : {
        "_index" : "index",
        "_id" : "id",
        "_source" : {
          "process" : {
            "thread" : {
              "id" : "9"
            }
          },
          "@timestamp" : "2020-10-22T20:28:26.267397Z",
          "message" : "set session",
          "event" : {
            "category" : "database",
            "action" : "query"
          }
        },
        "_ingest" : {
          "timestamp" : "2022-08-17T09:27:34.587465824Z"
        }
      }
    }
  ]
}

CodePudding user response：

The updated answer as below, seperating the individual processors:

POST /_ingest/pipeline/_simulate
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "dissect": {
          "field" : "message",
          "pattern" : "%{@timestamp}\t%{->} %{process.thread.id} %{event.action}\t%{message}"
        }
      },
      {
        "set": {
          "field": "event.category",
          "value": "database"
        }
      },
      {
          "lowercase": {
          "field": "event.action"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "message": "2020-10-22T20:28:26.267397Z\t   9 Query\tset session"
      }
    }
  ]
}

CodePudding user response：

You can use lowercase processor in same ingest pipeline as shown below:

{
    "pipeline": {
        "description": "_description",
        "processors": [
            {
                "dissect": {
                    "field": "message",
                    "pattern": "%{@timestamp}\t%{->} %{process.thread.id} %{event.action}\t%{message}"
                }
            },
            {
                "set": {
                    "field": "event.category",
                    "value": "database"
                }
            },
            {
                "lowercase": {
                    "field": "event.action"
                }
            }
        ]
    },
    "docs": [
        {
            "_index": "index",
            "_id": "id",
            "_source": {
                "message": "2020-10-22T20:28:26.267397Z\t   9 Query\tset session"
            }
        }
    ]
}