So I have a function, let's call it dostuff() in which it's beneficial for my application to sometimes parallelize within, or to do it multiple times and parallelize the whole thing. The function does not change though between both use cases.

Note: object is large enough that it cannot viably be stored in a list, and so it must be discarded with each iteration.

So, let's say our code looks like this:

bool parallelize_within = argv[1];
if (parallelize_within) {
    // here we assume parallelization is handled within the function dostuff()
    for (int i = 0; i < 100;   i) {
        object randomized = function_that_accesses_rand();
        dostuff(i, randomized, parallelize_within);
    }
} else {
#pragma omp parallel for
    for (int i = 0; i < 100;   i) {
        object randomized = function_that_accesses_rand();
        dostuff(i, randomized, parallelize_within);
    }
}

Obviously, we run into the issue that dostuff() will have threads access the random object at different times in different iterations of the same program. This is not the case when parallelize_within == true, but when we run dostuff() in parallel individually per thread, is there a way to guarantee that the random object is accessed in order based on the iteration? I know that I could do:

#pragma omp parallel for schedule(dynamic)

which will guarantee that eventually, as iterations are assigned to threads at runtime dynamically, the objects will access rand in order with the iteration number, but for the first set of iterations it will be totally random. Any suggestions on how to avoid this?

CodePudding user response：

You could pre generate the random object and store it in a list. Then have a variable in the omp loop, that is incremented per thread.

// generate random objects
i=0
#pragma omp parallel for
for( ... ){
  do_stuff(...,rand_obj[i],...)

CodePudding user response：

1. First of all you have to make sure that both function_that_accesses_rand and do_stuff are threadsafe.

2. You do not have to duplicate your code if you use the if clause: #pragma omp parallel for if(!parallelize_within)

3. To make sure that in function dostuff(i, randomized,...); i reflects the order of creation of randomized object you have to do something like this:

int j = 0;
#pragma omp parallel for if(!parallelize_within)
for (int i = 0; i < 100;   i) {
    int k;
    object randomized;
    #pragma omp critical
    {
        k = j  ;
        randomized = function_that_accesses_rand();        
    }
    dostuff(k, randomized, parallelize_within);
}

4. You may eliminate the use of the critical section if your function_that_accesses_rand makes it possible, but I cannot be more specific without knowing your function. One solution is that this function returns the value representing the order. Do not forget that this function has to be threadsafe!

#pragma omp parallel for if(!parallelize_within)
for (int i = 0; i < 100;   i) {
    int k;
    object randomized = function_that_accesses_rand(k);        
    dostuff(k, randomized, parallelize_within);
}

... function_that_accesses_rand(int& k){
 ...  
 #pragma omp atomic capture
 k = some_internal_counter  ;
 ...
}