I want to make a query in elasticsearch that returns the duplicates, but the query returns error 400 and set fieldata=True.

I need to make a query in elasticsearch,

I currently have a query:

{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "duplicateCount": {
      "terms": {
        "field": "hash_code_document",
        "min_doc_count": 2
      }
    }
  }
}

but when doing the query I get this 400 error:

{
 "error" : {
  "root_cause" : [
   {
    "type" : "illegal_argument_exception",
    "reason" : "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [hash_code_document] in order to load field data by uninverting the inverted index. Note that this can use significant memory."
   }
  ],
  "type" : "search_phase_execution_exception",
  "reason" : "all shards failed",
  "phase" : "query",
  "grouped" : true,
  "failed_shards" : [
   {
    "shard" : 0,
    "index" : "curriculo-19",
    "node" : "QOzYVehEQhezjq1TWxYvAA",
    "reason" : {
     "type" : "illegal_argument_exception",
     "reason" : "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [hash_code_document] in order to load field data by uninverting the inverted index. Note that this can use significant memory."
    }
   }
  ],
  "caused_by" : {
   "type" : "illegal_argument_exception",
   "reason" : "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [hash_code_document] in order to load field data by uninverting the inverted index. Note that this can use significant memory.",
   "caused_by" : {
    "type" : "illegal_argument_exception",
    "reason" : "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [hash_code_document] in order to load field data by uninverting the inverted index. Note that this can use significant memory."
   }
  }
 },
 "status" : 400
}

Do I need to change the mapping to make the query?

CodePudding user response：

Look like your field defined as "text" in the mapping. Elasticsearch doesn't allow to do the aggregations in this type of fields. You need to change the field type in the mapping to the "keyword" like:

 "mappings": {
    "properties": {
      "hash_code_document":{
        "type": "keyword"
      }
    }
  }

Or if you already have field like: hash_code_document.keyword, you need to use it for aggregation