Java array partitioning with variable input and thread count-CodePudding

I have an integer array as input, and its size can vary between 1 and 10^6:

eg.

int inputArr = new int[]{1, 2, 3, 4, 5};

I set the thread count dynamically at run time like so:

int threadCount = Runtime.getRuntime().availableProcessors() * 2;

The number of threads can vary from 4 to 64. Which means, the thread count might be even greater than the size of inputArr depending on the platform.

Since the inputArr can also become quite large, I would like to process the inputArr in parallel in small blocks.

So far I have managed to create the following

public class Main {
    public static void main(String[] args) {
        int threadCount = Runtime.getRuntime().availableProcessors() * 2;
        ExecutorService executor = Executors.newFixedThreadPool(threadCount);
        final var inputArr = new int[]{1, 2, 4, 5};

        for (int i = 0; i < threadCount; i  ) {
            executor.submit(new Worker(inputArr, i * threadCount, (i   1) * threadCount));
        }

        executor.shutdown();
        try {
            executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    private static class Worker implements Runnable {
        private final int[] arr;
        private final int start;
        private final int end;

        public Worker(int[] arr, int startIndex, int endIndex) {
            this.arr = arr;
            this.start = startIndex;
            this.end = endIndex;
        }

        @Override
        public void run() {
            System.out.printf("Thread start: %d\n", Thread.currentThread().getId());
            for (int i = this.start; i < this.end; i  ) {
                System.out.println(arr[i]);
            }
            System.out.printf("Thread end: %d\n", Thread.currentThread().getId());
        }
    }
}

And the output is similar to this:

...
Thread start: 27
Thread start: 34
Thread start: 33
Thread start: 24
1
2
4
5
Thread start: 30
Thread start: 26
...

Obviously, many threads are created here. Far more than the size of my inputArr in this example.

But instead, I would expect the output to look like this:

...
Thread start: X
1
2
4
5
Thread end: X
...

I think my problem is the wrong partitioning of the array. What am I doing wrong? Can someone help me please.

CodePudding user response：

Your partitioning does not look correct, yes.

Instead of

for (int i = 0; i < threadCount; i  ) {
  executor.submit(new Worker(inputArr, i * threadCount, (i   1) * threadCount));
}

Try this

final var batchSize = inputArr.length / threadCount == 0 ? inputArr.length : inputArr.length / threadCount;
final var remainder = inputArr.length % threadCount;
final int partitionSize = threadCount > batchSize ? 1 : threadCount;
for (int i = 0; i < partitionSize; i  ) {
    int endIndex = (i   1) * batchSize;
    executor.submit(new Worker(inputArr, i * batchSize, endIndex > inputArr.length ? remainder : endIndex));
}