Replacing values in a numpy array by averaging adjacent values-CodePudding

I have a 3D numpy array

dark = np.array(dark_ref.load())
dark.shape
(100, 384, 249)

I have a list of indices for the second and third dimension

l, r, b = np.where(dark > 9000)
len(r)
1799
len(b)
1799

and I have written this function to replace the values at indices r and b with an average of the adjacent values

def flatten_bad_band(A, x, y): # x is 3d array, y is bad coords
    testarr = A.copy()
    for a in range(A.shape[0]):
        for i in range(A.shape[1]):
            for j in range((A.shape[2])):
                testarr[a, x[i], y[j]] = (testarr[a, x[i 1], y[j]]   testarr[a, x[i-1], y[j]]) /2 
    return testarr
test = flatten_bad_band(dark, r, b)

This seems to work, but it is slow...., and I will need to use this on much larger arrays. Is there a better way to do this? I am new to python, numpy and coding!

CodePudding user response：

I have a 3D numpy array

dark = np.array(dark_ref.load())
dark.shape
(100, 384, 249)

I have a list of indices for the second and third dimension

l, r, b = np.where(dark > 9000)
len(r)
1799
len(b)
1799

and I have written this function to replace the values at indices r and b with an average of the adjacent values

def flatten_bad_band(A, x, y): # x is 3d array, y is bad coords
    testarr = A.copy()
    for a in range(A.shape[0]):
        for i, j in zip(x, y):
                testarr[a, x[i], y[j]] = (testarr[a, x[i 1], y[j]]   testarr[a, x[i-1], y[j]]) /2 
    return testarr
dark_clean = flatten_bad_band(dark, r, b)

commenter @Nyquist, suggested an improvement to skip one loop and pointed out an error that meant the code wasnt working as intended. fixed with his suggestion and seems to work faster too

CodePudding user response：

You can also eliminate the first for loop and let numpy do all the work along the axis. Don't know if this is faster, but since numpy has a C backend, it is more than likely that it is! It is also good practice.

Besides that, you should look into python's for loops! Your code still doesn't work as you attempt to index your list with elements of your list.

Here is an example, eliminating all unnecessary loops and also deals with cases where you attempt to average out of bounds elements. Hope this helps.

import numpy as np

Atest = np.random.randint(0,10, (5,5,5)).astype(float)

l = [1,2]
r = [2,3]

def flatten_bad_band(A, l, r):
    resarr = A.copy()
    testarr = np.pad(resarr, ((0,0),(1,1),(0,0)), 'constant', constant_values=0)
    
    for idx in zip(l, r):
        resarr[:,idx[0], idx[1]] = np.average(np.stack((testarr[:,idx[0], idx[1]], testarr[:,idx[0] 2, idx[1]])), axis=0)

    return resarr

print(flatten_bad_band(Atest, l, r))