NodeJS Performance with working with arrays-CodePudding

I want to retrieve a user's chats with corresponding users from a different collection in NodeJS and MongoDB.
The nature of NodeJS gives me a bad feeling that running the following code will block or decrease performance of my app. I can duplicate some data but I want to learn more about NodeJS.

Please let me know whether my code is ok and will not decrease performance.

Here I fetch 20 chats. I also need their corresponding users. then I get the userIds and perform another query against the User collection. Now I have both but I should merge them using Array.map.

I don't use $lookup because my collections are sharded.

$lookup

Performs a left outer join to an unsharded collection in the same database to filter in documents from the "joined" collection for processing. To each input document, the $lookup stage adds a new array field whose elements are the matching documents from the "joined" collection. The $lookup stage passes these reshaped documents to the next stage. https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/#mongodb-pipeline-pipe.-lookup

let chats = await Chat.find({ active: true }).limit(20);
/*
[
  {_id: ..., userId: 1, title: 'Chat A'},
  ...
]
*/


const userIds = chats.map(item => item.userId);
/*
[1, ...]
*/


const users = await User.find({ _id: { $in: userIds }});
/*
[
  {_id: 1, fullName: 'Jack'},
  ...
]
*/


chats = chats.map(item => {
  item.user = users.find(user => user._id === item.userId);

  return item;
});
/*
[
  {
    _id: ...,
    userId: 1,
    user: {_id: 1, fullName: 'Jack'},    // <-------- added
    title: 'Chat A'
  },
  ...
]
*/

CodePudding user response：

You are using async/await, so your code will wait a response from every time use await

// Wait to finish here
let chats = await Chat.find({ active: true }).limit(20);
/*
[
  {_id: ..., userId: 1, title: 'Chat A'},
  ...
]
*/

// Wait to finish here too
const users = await User.find({ _id: { $in: userIds }});
/*
[
  {_id: 1, fullName: 'Jack'},
  ...
]
*/

So if you has too many data and you don't have any index on your collection it will be too long to finish those query. At this case you should create ref in your collection Chat to collection User with chat.userId = user._id

Then when you call query chat, you populate field userId so you don't have to map const userIds = chats.map(item => item.userId); and chats = chats.map...

Sample for chat schema

const { Schema, model } = require("mongoose");

const chatSchema = new Schema({
  active: Boolean,
  userId: {
    type: "ObjectId",
    ref: "User",
  },
  title: String,
  message: String
  // another property
});

const userSchema = new Schema({
  username: String,
  email: String
  // another property
})


// query for chat

const chatModel = new model('chat', chatSchema)
let chats = await chatModel.find({ active: true }).populate('userId').limit(20);

/*
[
  {
    _id: ...,
    userId: {_id: 1, fullName: 'Jack'},    // <-------- already have
    title: 'Chat A'
  },
  ...
]
*/

CodePudding user response：

This is NOT how you should do it. MongoDB has something called Aggregation Framework and $lookup pipeline that will do that for you automatically with only 1 MongoDB query.

But since you are using Mongoose, this query become even more simpler since you can use populate() method of the Mongoose. So your whole code can be replaced with one line like this:

const chats = await Chat.find({ active: true }).populate('userId;).limit(20);

console.log(chats)

Note: If your collections are sharded, in my opinion you already implemented the logic in best possible way.