Home > Blockchain >  Java List Directory With Sorting Performance Issue
Java List Directory With Sorting Performance Issue

Time:10-17

I am building a file explorer where am sorting directories and files by name (case insensitive) and ordering directories before files, am using the following code but it is slow in terms of performance, so is there any other way to accomplish this:

File[] directories = new File(path).listFiles(File::isDirectory);
File[] files = new File(path).listFiles(File::isFile);

Arrays.sort(directories, Comparator.comparing(File::getName, String.CASE_INSENSITIVE_ORDER));
Arrays.sort(files, Comparator.comparing(File::getName, String.CASE_INSENSITIVE_ORDER));

File[] list = new File[directories.length   files.length];

System.arraycopy(directories, 0, list, 0, directories.length);  
System.arraycopy(files, 0, list, directories.length, files.length); 

CodePudding user response:

First of all, you should do benchmarking to figure out where is the exact bottleneck. How do I write a correct micro-benchmark in Java? should be a good start.

Now, an idea, which might help.

  1. Get all files and directories in a single array. Like this you are getting file and folders with a single operation, instead of two. Accessing file and folders on the disk is not exactly fast operation, so you would want to reduce those. Also if you take a look at the implementation of listFiles(FileFilter), it iterates everything in the folder to find matches, so that's 1 less iteration of all elements.
File[] directoriesAndFiles = folder.listFiles(file -> file.isDirectory() || file.isFile());
  1. Write composite comparator for the sort. Judging by code you want directories first.
public class FileTypeComparator implements Comparator<File> {

    @Override
    public int compare(File first, File second) {
        if (first.isDirectory() && second.isFile()) {
            return -1;
        }
        if (first.isFile() && second.isDirectory()) {
            return 1;
        }
        return 0;
    }
}

Combine it with your current comparator and do the sort:

Comparator<File> compositeComparator = new FileTypeComparator()
    .thenComparing(File::getName, String.CASE_INSENSITIVE_ORDER);
Arrays.sort(directoriesAndFiles, compositeComparator);

Like this you also don't need to copy initial arrays into the resulting one.

CodePudding user response:

If you have a huge number of files in a folder or are using non-SSD storage the calls to File.listFiles() and File.isDirectory (or isFile) can make the folder scan quite slow. Doing the scan in one step is possible but the sort by dir / file will still repeat the calls to isDirectory/isFile again.

Instead you should look at implementing with Files.find, which means you can read all the file attributes at the same time so that sort does not re-read file system attributes again. It is neatly handled in one stream. Here is an example which just prints the sorted items and modification times of the current folder:

public static Stream<Map.Entry<Path, BasicFileAttributes>>
find(Path dir, int maxDepth, BiPredicate<Path, BasicFileAttributes> matcher, FileVisitOption... options) throws IOException {

    // Using ConcurrentHashMap is safe to use with parallel()
    ConcurrentHashMap<Path,BasicFileAttributes> attrs = new ConcurrentHashMap<>();

    BiPredicate<Path, BasicFileAttributes> predicate = (p,a) -> (matcher == null || matcher.test(p, a)) && attrs.put(p, a) == null;
    return Files.find(dir, maxDepth, predicate, options).map(p -> Map.entry(p, attrs.remove(p)));
}

public static void main_sort_files(String[] args) throws IOException {
    Path dir = Path.of(args[0]);
    int depth = 1;

    // Note it is easy to add more sort fields here:
    Comparator<Entry<Path, BasicFileAttributes>> compDirs = Comparator.comparing(entry -> entry.getValue().isRegularFile());
    Comparator<Entry<Path, BasicFileAttributes>> comparing = compDirs.thenComparing(entry -> entry.getKey().getFileName().toString(), String.CASE_INSENSITIVE_ORDER);

    try(var files = find(dir, depth, (p,a) -> true)) {
        files.sorted(comparing).forEach(entry -> System.out.println(entry.getKey()  " modified " entry.getValue().lastModifiedTime()));

        // Alternatively get files folder Paths:
        // List<Path> contents = files.sorted(comparing).map(Entry::getKey).toList();
    }
}

It can be made into tree scan by editing depth to Integer.MAX_VALUE.

  • Related