Home > Enterprise >  Elasticsearch - Using copy_to on the fields of a nested type
Elasticsearch - Using copy_to on the fields of a nested type

Time:07-21

Elastic version 7.17

Below I've pasted a simplified version of my mappings which represent a nested object structure. One top-level-object will have one or more second-level-object. A second-level-object will have one or more third-level-object. Fields field_a, field_b, and field_c on third-level-object are all related to each other so I'd like to copy them into a single field that can be partial matched against. I've done this on a lot of attributes at the top-level-object level, so I know it works.

{
    "mappings": {
        "_doc": { //one top level object
            "dynamic": "false",
            "properties": {
                "second-level-objects": { //one or more second level objects
                    "type": "nested",
                    "dynamic": "false",
                    "properties": {
                        "third-level-objects": { //one or more third level objects
                            "type": "nested",
                            "dynamic": "false",
                            "properties": {
                                "my_copy_to_field": { //should have the values from field_a, field_b, and field_c
                                    "type": "text",
                                    "index": true
                                },
                                "field_a": {
                                    "type": "keyword",
                                    "index": false,
                                    "copy_to": "my_copy_to_field"
                                },
                                "field_b": {
                                    "type": "long",
                                    "index": false,
                                    "copy_to": "my_copy_to_field"
                                },
                                "field_c": {
                                    "type": "keyword",
                                    "index": false,
                                    "copy_to": "my_copy_to_field"
                                },
                                "field_d": {
                                    "type": "keyword",
                                    "index": true
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

However, when I run a nested query against that my_copy_to_field I get no results because the field is never populated, even though I know my documents have data in the 3 fields with copy_to. If I perform a nested query against field_d which is not part of the copied info I get the expected results, so it seems like there's something going on with nested (or double-nested in my case) usage of copy_to that I'm overlooking. Here is my query which returns nothing:

GET /my_index/_search
{
  "query": {
    "nested": {
      "inner_hits": {}, 
      "path": "second-level-objects",
      "query": {
        "nested": {
          "inner_hits": {}, 
          "path": "second-level-objects.third-level-objects",
          "query": {
            "bool": {
              "should": [
                {"match": {"second-level-objects.third-level-objects.my_copy_to_field": "my search value"}}  
              ]
            }
          }
        }
      }
    }
  }
}

I've tried adding include_in_root:true to the third-level-objects, but that didn't make any difference. If I could just get the field to populate with the copied data then I'm sure I can work through getting the query working. Is there something I'm missing about using copy_to with nested fields?

Additionally, when I view my data in Kibana -> Discover, I see second-level-objects as an available "Nested" field, but I don't see anything for third-level-objects, even though KQL recognizes it as a field. Is that symptomatic of an issue?

CodePudding user response:

You must add complete path nested, like this:

         "field_a": {
            "type": "keyword",
            "copy_to": "second-level-objects.third-level-objects.my_copy_to_field"
          },
          "field_b": {
            "type": "long",
            "copy_to": "second-level-objects.third-level-objects.my_copy_to_field"
          },
          "field_c": {
            "type": "keyword",
            "copy_to": "second-level-objects.third-level-objects.my_copy_to_field"
          }
  • Related