I have a Web application where a request may invoke multiple jobs, which have to run in parallel. Jobs involve communication over the network, so they are not exactly CPU-bound; estimated duration of such job is between several seconds and several minutes.

The obvious solution is to delegate job execution to a dedicated thread pool. However I would like to avoid the situation when one client (i.e. one request) hogs most of the pool for its own jobs leaving nothing to the others. At the same time, the job-creating request should not block; it should submit its jobs for execution and immediately return. Essentially what I am looking for is the ability to set quota based on some custom criterion, so that, for example, no more than 20 matching jobs would be running at the same time, even if more have been submitted to the thread pool. Is there perhaps anything out-of-the-box for this?

Another alternative of course is to create a new fixed thread pool instance for each request, submit jobs to it and then shutdown, but it does not feel like the right way to do it.

CodePudding user response：

I can't think of anything out of the box. However, some time ago I've implemented a solution solving similar problem, that could be adapted.

Just as an example, let's assume that you want to have a thread pool with total 200 threads available, but you want to allow only 10 threads to single identifier (in your case: request-id).

What you could do is to create 20 separate thread pools, each with 10 threads. Then have some simple logic that forwards task to one of those threadpools. It could be done i.e. based on hash-of-task-id%number-of-threadpools.

This would guarantee that all tasks from one request (or at least with the same request-id) would go to one specific executor.

Of course there are some assumptions here (i.e. hash of your id must be distributed well, so that when you get its modulo, it evenly distributes ids across threadpools), and risks (i.e. it's possible that tasks from multiple requests will go to one pool, while other pools may be empty).

If you needed more generic solution, then I'm afraid more implementation would be needed.

Here's the code for above mentioned solution:

public class IdAffiliatiedThreadPool {

    private final int executorsCnt;
    private final ExecutorService[] executors;
    
    public IdAffiliatiedThreadPool(String name, int executorsCnt, int threadsPerExecutor) {
        this.executorsCnt = executorsCnt;
        this.executors = new ExecutorService[executorsCnt];
        
        for(int i = 0; i < executorsCnt; i  ) {
            this.executors[i] = Executors.newFixedThreadPool(threadsPerExecutor);
        }
    }
    
    public void execute(Object id, Runnable action) {
        int threadId = id.hashCode() % executorsCnt;
        this.executors[threadId].execute(action);
    }
}