Windows threading synchronization performance issue-CodePudding

The (The first bar is the I/O thread reading data, the smaller bars are the process threads. Red: synchronization, green: process, purple: I/O)

This is from Visual Studio's concurrency visualizer, which tells me that 63% of the time was spent on thread synchronization. As far as I can tell, my code is the same for both the Linux and windows executions.

I made my best to make the objects immutable to avoid issues and that provided a big gain with my old 8-thread intel i7. However with many more threads, this issue arises.

For threading, I have tried a custom parallel for, and the

As you can see, almost all of the time is spent in malloc or free, which get called from the various Matrix operations. The bottom part of the image shows the timeline of the activity of a small selection of the threads: Green means that the thread is inactive, i.e. waiting. Usually only one or two threads are actually active. Allocations and freeing memory accesses a shared resource, causing the threads to wait for each other.

I think you have only two real options:

Option 1: No dynamic allocations anymore

The most efficient thing to do would be to rewrite the code to preallocate everything and get rid of all the temporaries. To adapt it to your

Option 2: Using a special allocator

The alternative is to use a different allocator that handles allocating and freeing memory more efficiently in multithreaded scenarios. One that I had very good experience with is mimalloc (there are others such as hoard or the one from TBB). You do not need to modify your source code, you just need to link with a specific library as described in the documentation.

I tried mimalloc with your source code, and it gave near 100% CPU utilization without any code changes. I also found a post on the Intel forums with a similar problem, and the solution there was the same (using a special allocator).

Additional notes

Matrix::allocSpace() allocates the memory by using pointers to arrays. It is better to use one contiguous array for the whole matrix instead of multiple independent arrays. That way, all elements are located behind each other in memory, allowing more efficient access.

But in general I suggest to use a dedicated linear algebra library such as Eigen instead of the hand rolled matrix implementation to exploit vectorization (SSE2, AVX,...) and to get the benefits of a highly optimized library.

Ensure that you compile your code with optimizations enabled.

Disable various cross-checks if you do not need them: assert() (i.e. define NDEBUG in the preprocessor), and for MSVC possibly /GS-.

Ensure that you actually have enough memory installed.

CodePudding user response：

You said that all your memory was pre-allocated, but in the worker function I see this...

Matrix b = input_data.getAt(t);

which allocates and fills a new matrix b, and this...

Matrix A(input_data.getDim(), input_data.getDim());

which allocates and fills a new matrix A, and this...

Matrix x = A * b;

which allocates and fills a new matrix x.

The heap is a global data structure, so the thread synchronization time you're seeing is probably contention in the memory allocate/free functions.

These are in a tight loop. You should fix this loop to access b by reference, and reuse the other 2 matrices for every iteration.