In my case, NIFI will receive data from syslog firewall, then after transformation sends JSON to ELASTIC. This is my first contact with ELASTICSEARCH

{   
"LogChain" : "Corp01 input",   
"src_ip" : "162.142.125.228",   
"src_port" : "61802",   
"dst_ip" : "177.16.1.13",   
"dst_port" : "6580",   
"timestamp_utc" : 1646226066899 
}

In Elasticsearch automatically created Index with such types

{
  "mt-firewall" : {
    "mappings" : {
      "properties" : {
        "LogChain" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "dst_ip" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "dst_port" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "src_ip" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "src_port" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "timestamp_utc" : {
          "type" : "long"
        }
      }
    }
  }
}

How to change type fields in Elasticsearch?

• "src_ip": type "ip"
• "dst_ip": type "ip"
• "timestamp_utc": type "data"

CodePudding user response：

You can change or configure field type using Mapping in Elasticsearch and some of the way i have given below:

1. Explicit Index Mapping

Here, you will define index mapping by your self with all the required field and specific type of field before indexing any document to Elasticsearch.

PUT /my-index-000001
{
  "mappings": {
    "properties": {
      "src_ip":    { "type": "ip" },  
      "dst_ip":  { "type": "ip"  }, 
      "timestamp_utc":   { "type": "date"  }     
    }
  }
}

2. Dyanamic Template:

Here, you will provide dynamic template while creating index and based on condition ES will map field with specific data type like if field name end with _ip then map field as ip type.

PUT my-index-000001/
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_ip": {
          "match_mapping_type": "string",
          "match": "*ip",
          "runtime": {
            "type": "ip"
          }
        }
      }
    ]
  }
}

Update 1:

If you want to update mapping in existing index then it is not recommndate as it will create data inconsistent.

You can follow bellow steps:

1. Use Reindex API to copy data to temp index.
2. Delete your original index.
3. define index with one of the above one method with index mapping.
4. Use Reindex API to copy data from temp index to original index (newly created index with Mapping)