Having switched from Elasticsearch to Opensearch, my application now fails to run a simple query with:

"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [status] in order to load field data by uninverting the inverted index. Note that this can use significant memory."

There's a question concerning the same error at Searchkick / Elasticsearch Error: Please use a keyword field instead. Alternatively, set fielddata=true on [name], but there the problem was only affecting tests and I only get the problem (so far) in development mode.

Here's the query being run:

::Record.search(q ? q : "*",
                where: where_clause,
                fields: fields,
                match: :word_middle,
                per_page: max_per_page(per_page) || 30,
                page: page || 1,
                order: sort_clause,
                aggs: aggs,
                misspellings: {below: 5}

If I take out aggs then the search is fine, but they're essential for the application. Removing :status from the list of aggregation fields causes the error to name the next field in the array as the problem. So, I presumably need to specify the correct type for each field used in aggregations. But how?

The Searchkick docs suggest this example under "Advanced Mapping" (https://github.com/ankane/searchkick):

class Product < ApplicationRecord
  searchkick mappings: {
    properties: {
      name: {type: "keyword"}
    }
  }
end

So, I tried this:

# in models/Record.rb
mapping_properties = {}
aggregation_fields.each do |af|
  mapping_properties[af] = { type: 'keyword' }
end
searchkick mappings: {
             properties: mapping_properties
          }

But, the same problem persists. I also tried something similar to that shown in the linked post, e.g.

mappings: {
      properties: {
        name: {
          type: "text",
          fielddata: true,
          fields: {
            keyword: {
              type: "keyword"
            }
          }
        }
      }
    }

...but similarly without luck.

Can anyone suggest how this might be fixed?

CodePudding user response：

The immediate issue was dealt with by changing all the fields used for aggregations, so rather than:

aggs = %w(field1 field2 field3 ...)

...in the above search query. I used:

aggs = %w(field1.keyword field2.keyword field3.keyword ...)

CodePudding user response：

This was not related to using Opensearch. Instead, the method of indexing used caused the indexes to be generated incorrectly.

See: Searchkick memory leak