Elasticsearch nested object mapping doesn't work correctly-CodePudding

I'm trying to set up an elasticsearch index with an array of objects. I tried the following mapping:

{
    "mappings": {
    "date_detection": false,
    "properties": {
    "resource": {
        "type": "object",
        "properties": {
          "name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "uid": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "id": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "source": {
            "properties": {
              "serviceType": {
                "type": "text"
              },
              "serviceId": {
                "type": "text"
              },
              "state": {
                "type": "text"
              },
              "type": {
                "type": "text"
              },
              "connectorName": {
                "type": "text"
              },
              "displayName": {
                "type": "text"
              }
            }
          },
          "_key": {
            "type": "text"
          }
        }
      },
    // other, irrelevnt fields
    }
}
}

And putting the following document:

"resource": [
  {
    "source": {
      "serviceType": "AWS",
      "serviceId": "...",
      "state": null,
      "type": "Source",
      "connectorName": "AWS",
      "displayName": null
    },
    "name": "...",
    "id": "...",
    "_key": "...",
    "uid": "..."
  },
  {
    "source": {
      "serviceType": "AWS",
      "serviceId": "..",
      "state": null,
      "type": "Source",
      "connectorName": "AWS",
      "displayName": null
    },
    "name": "...",
    "id": "...",
    "_key": "...",
    "uid": "..."
  }

However it seems like the resource field is being parsed correctly:

I tried playing around with the fields but didn't manage to make it work. What am I missing?

CodePudding user response：

You missed out on the properties key in your index mapping. The correct index mapping should be :

{
    "mappings": {
        "properties": {                                           // note this
            "resource": {
                "type": "object",
                "properties": {
                    "name": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "uid": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "id": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "source": {
                        "properties": {
                            "serviceType": {
                                "type": "text"
                            },
                            "serviceId": {
                                "type": "text"
                            },
                            "state": {
                                "type": "text"
                            },
                            "type": {
                                "type": "text"
                            },
                            "connectorName": {
                                "type": "text"
                            },
                            "displayName": {
                                "type": "text"
                            }
                        }
                    },
                    "_key": {
                        "type": "text"
                    }
                }
            }
        }
    }
}

CodePudding user response：

I found the issues. First, it was a conflicting field, not unknown - I had to make sure the index pattern was only containing the new index I was working with. Then, I faced the issue described in Kibana reports a field is conflicting, how can I resolve it? . Lastly, it stayed as "unknown" field, until I created a brand new index name with a new index pattern, and it resolved.