I have a nodejs app that has one getUsers endpoint, this endpoint gets the first 10 users from a mongodb database and returns them in a JSON response.
Lets say this is a sync process (async wont make any sense here since i need to fetch and return the result in the same request) and the entire process takes 1 sec. So, if I have 4 CPU in a virtual machine and I am running the nodejs in cluster mode (meaning 4 nodejs processes running together).
Q1) Does this mean my app can handle 4 simultaneous request per second? If this is true, how can my app handle 1000 request per second?
Q2) I know nodejs is single threaded but Is there anyway the nodejs instance can use more than 1 thread per CPU so that it can handle more request per cpu per second?
CodePudding user response：
Q1) Does this mean my app can handle 4 simultaneous request per second? If this is true, how can my app handle 1000 request per second?
Yes, not because of multiple instances but because Nodejs handles each incoming request asynchronously, even though the processing of a request is itself synchronous. You can handle the 4 of them with a single instance.
The number of requests that can be handled depends on your server.
I recommend doing a performance test. For example, I have recently used Grafana K6
Q2) I know nodejs is single threaded but Is there anyway the nodejs instance can use more than 1 thread per CPU so that it can handle more request per cpu per second?
No, if you want use more CPU you have to create multiple instances in cluster mode.
CodePudding user response：
Q1) Does this mean my app can handle 4 simultaneous request per second?
If we neglect load balancer process time, your cluster could handle around 4 RPS (requests per second).
If this is true, how can my app handle 1000 request per second?
In short, by scaling via threads or server instances.
More details will be described below.
Q2) I know nodejs is single threaded
Falsy statement.
Node.JS has at least 4 threads to handle I/O operations and few more threads for other under the hood processes.
This makes even single Node.JS instance pretty powerful to handle many async operations.
but Is there anyway the nodejs instance can use more than 1 thread per CPU
Yes, you can use worker_thread to create as many threads as you wish.
so that it can handle more request per cpu per second?
Firstly, you have to make your request handler works asynchronous so it can process more than one request at a time within single thread (using single CPU core).
Then you would like to scale it up via worker_threads to use more threads i.e. more cores to process higher amount of requests.
And that's all inside single process.
P.S. Since each new thread will use new core, there exists a recommendation to spawn around N - 1 threads where N is number of logical cores on your CPU.