I have a thread pool on 8 thread

private static final ExecutorService SERVICE = Executors.newFixedThreadPool(8);

My mechanism emulating the work of 100 user (100 Tasks):

List<Callable<Boolean>> callableTasks = new ArrayList<>();
for (int i = 0; i < 100; i  ) { // Number of users == 100
    callableTasks.add(new Task(client));
}
SERVICE.invokeAll(callableTasks);
SERVICE.shutdown();

The user performs the Task of generating a document.

Get UUID of Task;
Get Task status every 10 seconds;
If Task is ready get document.

public class Task implements Callable<Boolean> {

    private final ReportClient client;

    public Task(ReportClient client) {
        this.client = client;
    }

    @Override
    public Boolean call() {
        final var uuid = client.createDocument(documentId);
        GetStatusResponse status = null;
        do {
            try {
                Thread.sleep(10000); // This stop current thread, but not a Task!!!!
            } catch (InterruptedException e) {
                return Boolean.FALSE;
            }
            status = client.getStatus(uuid);
        } while (Status.PENDING.equals(status.status()));
        final var document = client.getReport(uuid);
        return Boolean.TRUE;
    }
}

I want to give the idle time (10 seconds) to another task. But when the command Thread.sleep(10000); is called, the current thread suspends its execution. First 8 Tasks are suspended and 92 Tasks are pending 10 seconds. How can I do 100 Tasks in progress at the same time?

CodePudding user response：

EDIT: I just posted this answer and realized that you seem to be using that code to emulate real user interactions with some system. I would strongly recommend just using a load testing utility for that, rather than trying to come up with your own. However, in that case just using a CachedThreadPool might do the trick, although probably not a very robust or scalable solution.

Thread.sleep() behavior here is working as intended: it suspends the thread to let the CPU execute other threads. Note that in this state a thread can be interrupted for a number of reasons unrelated to your code, and in that case your Task returns false: I'm assuming you actually have some retry logic down the line.

So you want two mutually exclusive things: on the one hand, if the document isn't ready, the thread should be free to do something else, but should somehow return and check that document's status again in 10 seconds.

That means you have to choose:

You definitely need that once-every-10-seconds check for each document - in that case, maybe use a cachedThreadPool and have it generate as many threads as necessary, just keep in mind that you'll carry the overhead for numerous threads doing virtually nothing.
Or, you can first initiate that asynchronous document creation process and then only check for status in your callables, retrying as needed.

Something like:

public class Task implements Callable<Boolean> {
    private final ReportClient client;
    private final UUID uuid;
    // all args constructor omitted for brevity
    @Override
    public Boolean call() {
        GetStatusResponse status = client.getStatus(uuid);
        if (Status.PENDING.equals(status.status())) {
            final var document = client.getReport(uuid);
            return Boolean.TRUE;
        } else {
            return Boolean.FALSE; //retry next time
        }
    }
}

List<Callable<Boolean>> callableTasks = new ArrayList<>();
for (int i = 0; i < 100; i  ) { 
    var uuid = client.createDocument(documentId); //not sure where documentId comes from here in your code
    callableTasks.add(new Task(client, uuid));
}

List<Future<Boolean>> results = SERVICE.invokeAll(callableTasks); 
// retry logic until all results come back as `true` here

This assumes that createDocument is relatively efficient, but that stage can be parallelized just as well, you just need to use a separate list of Runnable tasks and invoke them using the executor service. Note that we also assume that the document's status will indeed eventually change to something other than PENDING, and that might very well not be the case. You might want to have a timeout for retries.

CodePudding user response：

The Answer by Yevgeniy looks correct, regarding Java today. You want to have your cake and eat it too, in that you want a thread to sleep before repeating a task but you also want that thread to do other work. That is not possible today, but may be in the future.

Project Loom

In current Java, a Java thread is mapped directly to a host OS thread. In all common OSes such as macOS, BSD, Linux, Windows, and such, when code executing in a host thread blocks (stops to wait for sleep, or storage I/O, or network I/O, etc.) the thread too blocks. The blocked thread suspends, and the host OS generally runs another thread on that otherwise unused core. But the crucial point is that the suspended thread performs no further work until your blocking call to sleep returns.

This picture may change in the not-so-distant future. Project Loom seeks to add virtual threads to the concurrency facilities in Java.

In this new technology, many Java virtual threads are mapped to each host OS thread. Juggling the many Java virtual threads is managed by the JVM rather than by the OS. When the JVM detects a virtual thread’s executing code is blocking, that virtual thread is "parked", set aside by the JVM, with another virtual thread swapped out for execution on that "real" host OS thread. When the other thread returns from its blocking call, it can be reassigned to a "real" host OS thread for further execution. Under Project Loom, the host OS threads are kept busy, never idled while any pending virtual thread has work to do.

This swapping between virtual threads is highly efficient, so that thousands, even millions, of threads can be running at a time on conventional computer hardware.

Using virtual threads, your code will indeed work as you had hoped: A blocking call in Java will not block the host OS thread. But virtual threads are experimental, still in development, scheduled as a preview feature in Java 19. Early-access builds of Java 19 with Loom technology included are available now for you to try. But for production deployment today, you'll need to follow the advice in the Answer by Yevgeniy.

Take my coverage here with a grain of salt, as I am not an expert on concurrency. You can hear it from the actual experts, in the articles, interviews, and presentations by members of the Project Loom team including Ron Pressler and Alan Bateman.