Unable to search on whole database with searchkick as it limits to 10000 records-CodePudding

Unable to search on whole elastic search DB just by using

SearchData.search('yamaha', match: :word_middle,load: false)

This limits the search to 10000 records but in my DB there are more than a hundred thousand records so, how to search on the whole DB not just the first ten thousand records I'm not able to find anything a little help will be appreciated

CodePudding user response：

I am not sure about searckick but Elasticsearch always search on entire index and not only 10000 recored but by default it returns only 10 documents in response. You can change response size by changing size parameters and max it return 10000 documents per request.

Also, if you have large index then it always show 10000 for hits.total.value in response and for getting actual number count you can set track_total_hits value to true.

{
  "track_total_hits": true,
  "query": {
    "match" : {
      "user.id" : "elkbee"
    }
  }
}

If you need to get more then 10000 documents from the Elasticsearch then you can use search_after or scroll API. you can refer this documentation for more details.

CodePudding user response：

Deep Paging

By default, Elasticsearch and OpenSearch limit paging to the first 10,000 results. Here’s why. We don’t recommend changing this, but if you really need all results, you can use:

class Product < ApplicationRecord
  searchkick deep_paging: true
end

If you just need an accurate total count, you can instead use:

Product.search("pears", body_options: {track_total_hits: true})