invokeAll but CompletableFuture-CodePudding

My task is to load file asynchronously using ExecutiveService & CompletableFuture, and to measure execution time for 2, 4, 8 threads and without parallelization

It`s my first method:

File folderWithJson = new File(pathToFolderWithJson);
ExecutorService executorService = Executors.newFixedThreadPool(16);


Set<Callable<Boolean>> callables = new HashSet<>();
for(File file: Objects.requireNonNull(folderWithJson.listFiles())) {
    callables.add(() -> {
        System.out.println(Thread.currentThread().getName());
        return getFineToStat(file);
    });
}
executorService.invokeAll(callables);
executorService.shutdown();

And it works finem but this way, i don`t use CompletableFuture, i have tried this:

File folderWithJson = new File(pathToFolderWithJson);
ExecutorService executorService = Executors.newFixedThreadPool(16);

for(File file: Objects.requireNonNull(folderWithJson.listFiles())) {
    CompletableFuture.runAsync(() -> {
        try {
            getFineToStat(file);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }, executorService).join();
}
executorService.shutdown();

But i get almost the same time with different number of threads, although i have 31 file totaling 850mb, there should be a difference

How do i implement invokeAll but using CompletableFuture

CodePudding user response：

.join() waits for the result of each CompletableFuture before even submitting the next. Here's an alternative:

    final ExecutorService executorService = Executors.newFixedThreadPool(16);
    //try with resources and using a Stream over the files (`Path`s) in the directory:
    try(Stream<Path> paths = Files.list(folderWithJson.toPath())) {
        final CompletableFuture<?>[] all = paths
               //each Path is mapped to a CompletableFuture, to be run on the ExecutorService:
               .map(path -> CompletableFuture.runAsync(() -> {
                   try {
                       getFineToStat(path.toFile());
                   } catch (IOException e) {
                       throw new RuntimeException(e);
                   }
               }, executorService))
               //we collect them into a new array, so that we can use them later
               .toArray(CompletableFuture[]::new);
        //this will wait for all to finish:
        CompletableFuture.allOf(all).join();
        executorService.shutdown();
    }

Please be aware that the entire process may abort if just one of the tasks completes with an error. To handle errors, avoid throwing the RuntimeException within the task itself. That way each file can be handled separately, and the process may be allowed to continue (or not, as you wish) even if an individual file has issues.