How can I add "Fork-join" to my file search code?-CodePudding

I wrote a program that searches for the necessary files in a directory and its subfolders based on the desired file size. But according to the assignment, I need to add a multithreaded "Fork-join" mode. I understand that he can slow down, not speed up the work, but this was my assignment. I can't add it to the code in any way to make everything work.

import java.io.File;
import java.text.DecimalFormat;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Scanner;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;

public class  Main{
    static Scanner in = new Scanner(System.in);
    static File directory = new File(in.next());
    public static final double MIN_LENGTH = in.nextDouble() * 1048576;  
    static int numOfThreads = 2; 



    public static void main(String[] args) {
        ArrayList<File> fileList = new ArrayList<>();
        getFiles(directory, fileList);
        fileList.sort(Comparator.<File>comparingLong(File::length).reversed());
        DecimalFormat df = new DecimalFormat("#.#");
        ForkJoinPool pool = new ForkJoinPool(numOfThreads);



        for(File file: fileList) {

            if(file.length()>= 1073741824) {
                double Gb = (double) file.length() / 1073741824;
                System.out.printf("%s Gb  %s%n", df.format(Gb), file); 
            } else if(file.length()>= 1048576) {
                System.out.printf("%s Mb  %s%n", file.length() / 1048576, file); 
            } else if (file.length()>= 1024){
                System.out.printf("%s Kb  %s%n", file.length() / 1024, file);  
            } else if (file.length()< 1024){
                System.out.printf("%s B  %s%n", file.length(), file); 
            } else {
                System.out.printf("%s Gb  %s%n", file.length() / (1073741824 * 1024), file); 
            }
        }
    }

    private static void getFiles(File rootFile, List<File> fileList) {
        if (rootFile.isDirectory()) {
            File[] directoryFiles = rootFile.listFiles();
            if (directoryFiles != null) {
                for (File file: directoryFiles) {
                    if (file.isDirectory()) {
                        getFiles(file, fileList);
                    } else {
                        if (file.isFile() && file.length() >= MIN_LENGTH) {
                            fileList.add(file);
                        }
                    }
                }
            }
        }
    }
}

I tried to divide the work into 2 threads, the first thread processes files up to 5 megabytes, and the second one connects if the file is larger than 5 megabytes, but everything turned out crookedly and does not work

CodePudding user response：

ForkJoinPool is poorly suited to scenarios where a thread is liable to block in an I/O operation. The problem is that the (Java 17) pool implementation cannot deal with an I/O operation that blocks in a syscall. As the javadoc says:

"For applications that require separate or custom pools, a ForkJoinPool may be constructed with a given target parallelism level; by default, equal to the number of available processors. The pool attempts to maintain enough active (or available) threads by dynamically adding, suspending, or resuming internal worker threads, even if some tasks are stalled waiting to join others. However, no such adjustments are guaranteed in the face of blocked I/O or other unmanaged synchronization. ".

CodePudding user response：

I think the idea is that each task should process a single file/directory.

Example:

    ForkJoinPool pool = new ForkJoinPool(20);
    FileTask task = new FileTask(directory);
    pool.invoke(task);

class FileTask:

public class FileTask extends RecursiveAction {
    private final DecimalFormat df = new DecimalFormat("#.#");
    private final File file;

    public FileTask(File file) {
        this.file = file;
    }

    @Override
    protected void compute() {
        if (file.isDirectory()) {
            processDir();
        } else if (file.isFile()) {
            processFile();
        }
    }

    private void processFile() {
        if(file.length()>= 1073741824) {
            double Gb = (double) file.length() / 1073741824;
            System.out.printf("%s Gb  %s%n", df.format(Gb), file); 
        } else if(file.length()>= 1048576) {
            System.out.printf("%s Mb  %s%n", file.length() / 1048576, file); 
        } else if (file.length()>= 1024){
            System.out.printf("%s Kb  %s%n", file.length() / 1024, file);  
        } else if (file.length()< 1024){
            System.out.printf("%s B  %s%n", file.length(), file); 
        } else {
            System.out.printf("%s Gb  %s%n", file.length() / (1073741824 * 1024), file); 
        }
    }

    private void processDir() {
        List<FileTask> tasks = new ArrayList<>();
        File[] subFiles = file.listFiles();
        for (File sub: subFiles) {
            tasks.add(new FileTask(sub));
        }
        ForkJoinTask.invokeAll(tasks);
    }
}