Speed up MongoDB Query Response | Discord Bot-CodePudding

I didn't think this would be an issue, but as the database I am using continues to grow, I've noticed that the responses from my program are getting slower and slower (which makes sense, more stuff to look through to return).

Now my question is how would I be able to speed up the performance of my response because it takes quite some time to do the following as of right now: I want to sort and return to the top 10 results from a query search, currently, my code looks something along the lines of:

alldocuments = self.bot.db.nameofdb.find()
alldocuments = list(map(lambda x: x["queryparam"], alldocuments ))
_documentCount = [(i, alldocuments.count(i)) for i in set(alldocuments)]
_documentCount.sort(key = lambda x: x[1], reverse=True)
docCount = list(filter(lambda x: ctx.guild.get_member(int(x[0])), _documentCount))
        
        
i = 0
while i < 10:
     if not ctx.guild.get_member(int(docCount[i][0])):
          i =1
          continue
     print(docCount[i][1]})
     i  = 1

Some information:

Database consists of thousands of entries as follow:

"name" : "Adam"
"location" : "America"
"Age": 20

Now say I want to get the most common name in America and the amount of times it appears. If the name Adam was in the database the most times 100k times, it should return Adam with the number of Adams in the db.

CodePudding user response：

Using MongoDB, you're able to directly include those filters that you are applying into your query. This should impact your runtime by loading fewer data.

documents = self.bot.db.nameofbd.find({"queryparam": True})
sorted_documents = documents.sort("queryparam": pymongo.DESCENDING)

CodePudding user response：

It's a bit difficult to give an answer without proper information (i.e. maybe 2-3 more sample input documents and sample result). However, could be this one:

db.collection.aggregate([
  {
    $group: {
      _id: {
        location: "$location",
        name: "$name"
      },
      count: { $sum: 1 }
    }
  },
  { $sort: { count: -1 } },
  { $limit: 10 }
])