How to correctly implement executor that runs multiple iterations and waits for all tasks to complet-CodePudding

Cut to the chase short answer ---------------------

Code demonstrating the accepted answer can be found here:

Full example:

https://github.com/NACHC-CAD/thread-example/tree/shutdown-first

Implementation:

https://github.com/NACHC-CAD/thread-example/blob/shutdown-first/src/main/java/com/nachc/examples/threadexample/WidgetFactory.java

Original Post -------------------------------------

There are a number of examples of use of Java threads and Executors: https://www.baeldung.com/thread-pool-java-and-guava

https://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html

https://howtodoinjava.com/java/multi-threading/java-thread-pool-executor-example/

https://jenkov.com/tutorials/java-concurrency/thread-pools.html

https://xperti.io/blogs/thread-pools-java-introduction/

https://www.journaldev.com/1069/threadpoolexecutor-java-thread-pool-example-executorservice

https://stackify.com/java-thread-pools/

However, I've not been able to successfully write an example that executes all of the tasks, waits for the tasks to complete, and then correctly terminates.

Working from this example: https://howtodoinjava.com/java/multi-threading/java-thread-pool-executor-example/

The code only calls executor.shutdown(). This does not allow the threads time to complete if they consume any time.

I've created a complete simplest example here: https://github.com/NACHC-CAD/thread-example/tree/await-termination

The shutdown only branch covers this use case (https://github.com/NACHC-CAD/thread-example/tree/shutdown-only):

public void makeWidgets() {
    ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(batchSize);
    log.info("Building "   howMany   " widgets...");
    for (int i = 0; i < howMany; i  ) {
        Widget widget = new Widget(lotNumber, i);
        WidgetRunnable runnable = new WidgetRunnable(widget);
        executor.execute(runnable);
    }
    log.info("SHUTTING DOWN----------------");
    executor.shutdown();
}

This code gives the following output (there should be 1000 widgets created and they should report that they are done after waiting 1 second).

2022-04-23 21:27:05,796 21:27:05.796 [main] INFO  (WidgetFactoryIntegrationTest.java:12) - Starting test...
2022-04-23 21:27:05,799 21:27:05.799 [main] INFO  (WidgetFactory.java:29) - Building 100 widgets...
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-2] INFO  (Widget.java:24) - Starting build: 1/1
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-4] INFO  (Widget.java:24) - Starting build: 1/3
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-1] INFO  (Widget.java:24) - Starting build: 1/0
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-5] INFO  (Widget.java:24) - Starting build: 1/4
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-6] INFO  (Widget.java:24) - Starting build: 1/5
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-7] INFO  (Widget.java:24) - Starting build: 1/6
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-8] INFO  (Widget.java:24) - Starting build: 1/7
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-10] INFO  (Widget.java:24) - Starting build: 1/9
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-9] INFO  (Widget.java:24) - Starting build: 1/8
2022-04-23 21:27:05,801 21:27:05.801 [main] INFO  (WidgetFactory.java:35) - SHUTTING DOWN----------------
2022-04-23 21:27:05,800 21:27:05.800 [pool-1-thread-3] INFO  (Widget.java:24) - Starting build: 1/2
2022-04-23 21:27:05,801 21:27:05.801 [main] INFO  (WidgetFactoryIntegrationTest.java:18) - Done.

If I add executor.awaitTermination the code runs all threads but never terminates. This example is in the await-termination branch: https://github.com/NACHC-CAD/thread-example/tree/await-termination

public void makeWidgets() {
    ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(batchSize);
    log.info("Building "   howMany   " widgets...");
    for (int i = 0; i < howMany; i  ) {
        Widget widget = new Widget(lotNumber, i);
        WidgetRunnable runnable = new WidgetRunnable(widget);
        executor.execute(runnable);
    }
    try {
        executor.awaitTermination(1000, TimeUnit.HOURS);
    } catch(Exception exp) {
        throw(new RuntimeException(exp));
    }
    log.info("SHUTTING DOWN----------------");
    executor.shutdown();
}

This code lets all of the runnables finish but never exits. How do I let all of the runnables finish and have the code run to completion (exit)?

CodePudding user response：

With reference to ThreadPoolExecutor documentation. The awaitTermination() method description reads:

Blocks until all tasks have completed execution after a shutdown request

While the shutdown() method descriptin reads

Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted

Which indicates that awaitTermination() call is effective after a shutdown() call.

To solve the above problem, shutdown() needs to be called first and then awaitTermination()

NOTE: I have not personally tested this; however, John has, as mentioned in the comment of the original post and the mechanism works