Home > Enterprise >  Issues with Mapping Elastic Index fields with Java Fields
Issues with Mapping Elastic Index fields with Java Fields

Time:11-09

Current Im working on a spring boot project to fetch the data from specific Elastic search index using Spring Data ElasticSearch (version 4.1.7). Im able to connect to elastic search and fetch all the data associated with a specific index. But there are some fields in the index which are returning null value. Where as the actual value exist in the index in elastic search .
Im having issues in retriving the specific fields due to incorrect field types. I have created a Document Class which

@Document(indexName = "netstat*")
@Setting(settingPath = "static/es-settings.json")
public class NetworkStats {
@Id
@Field(type = FieldType.Keyword)
private String id; 

@Field
private String timeStamp ;

@Field(type = FieldType.Text)
private String podName ; 

@Field(type = FieldType.Text)
private String iporHost ;

@Field(type = FieldType.Auto)
private String connectionCount ; 

@Field(type = FieldType.Text)
private String command ;    
}

Apart from the above I have getters & setters for the field as well defined.Im unabel to retrieve the values for fields connectionCount, iporHost, podName, timeStamp. Though the fields have values in the elastic index.
Sample data from ElasticSearch below

{
            "_index": "netstat-2021.11.09",
            "_type": "_doc",
            "_id": "0sJmA30BW7LXXVrK0nCg",
            "_score": 1.0,
            "_source": {
                "podname": "logindeployment-5479f7-4f2qr",
                "input": {
                    "type": "log"
                },
                "iporhost": "6200",
                "tags": [],
                "command": "OUTESTAB",
                "agent": {
                    "version": "7.15.1",
                    "id": "e8feeb4e-1f9e-41b2-bcba-38e507d176db",
                    "type": "filebeat",
                    "hostname": "BL1726L",
                    "name": "BL1726L",
                    "ephemeral_id": "c2859ece-3e95-4204-909d-af59322b9fa2"
                },
                "timestamp": "05-10-21T13:17:25.014719626",
                "ecs": {
                    "version": "1.11.0"
                },
                "connectionscount": "2",
                "@timestamp": "2021-11-09T06:31:29.104Z",
                "log": {
                    "offset": 1002,
                    "file": {
                        "path": "D:\\elastic_stack\\data\\log\\netStats.log"
                    }
                }
            }
        },

Require help on how to retrieve the data for the above mentioned fields.I guess there is some issue with field mapping. Any help on this is much appreciated.

CodePudding user response:

the fields in Elasticsearch seem all to have lower case names. Your properties have camel case. So you need to define the name:

@Field(type = FieldType.Text, name ="podname")
private String podName ; 

If you do not define a FieldType or use FieldType.Auto, then no mapping is written and Elasticsearch will try to guess the type. You should define the correct type to make sure that the conversion is done as desired.

Especially for the timestamp field you should define a date format that matches what you have in the index.

As the index exists you should make sure that you use the same type in the entity definition that is already used in the index.

  • Related