how to avoid loops when using numpy-CodePudding

I have an numpy array with size of 10x250000 and I need to change values of it during the program but using "for loop" make my program very slow. it is the mentioned section of program. (L2 , L1 and T are other arrays) :

EEf1=numpy.zeros((10,250000))
for m in range(10):
          for n in range(250000):
                    EEf1[m,n] =2*(L2[m,0]-T[m,0])*L2[m,0]*(1-L2[m,0])*L1[n,0]

My question : Is there any solution to use numpy features to avoid this loop ? I have tried fromfunction method like this (for example):

np.fromfunction(lambda i,j : A[i,j] B[i,j] , (2,2))

(A and B are 2x2 arrays) but it doesnt work. I really need to avoid this loop. can anyone help ?

CodePudding user response：

I think it's just broadcasting, but it's difficult to test without a reproducible example:

EEf1 = 2*(L2[:, 0] - T[:, 0]) * L2[:, 0] * (1 - L2[:, 0]) * L1[None, :, 0]

CodePudding user response：

Most of the time there is little change to do, just skip the for loops.

Here you have a shape mismatch between L1 and L2 -- np.outer does the job here:

def my_func(L1, L2, T):
    return np.outer(2 * (L2 - T) * L2 * (1 - L2), L1)

Also, it is a bit odd that L1, L2 and T are not 1D arrays to begin with, because you only ever use their first column.

Equality check:

M, N = 10, 25_000
L1 = np.random.rand(N, 1)
L2 = np.random.rand(M, 1)
T = np.random.rand(M, 1)

expected = op_func(L1, L2, T)
result = my_func(L1, L2, T)

np.array_equal(result, expected)  # True

Performance tests:

>>> %timeit op_func(L1, L2, T)
... %timeit my_func(L1, L2, T)

250 ms ± 2.24 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
85.2 µs ± 191 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

Your lines of code as a function, for reference:

def op_func(L1, L2, T):
    N, M = L1.shape[0], L2.shape[0]
    assert T.shape[0] == M
    EEf1 = np.zeros((M, N))
    for m in range(M):
        for n in range(N):
            EEf1[m, n]  = (
                2 * (L2[m, 0] - T[m, 0]) * L2[m, 0] * (1 - L2[m, 0]) * L1[n, 0]
            )
    return EEf1