Home > other >  Numpy way to temporarily remove NaN values from array, with ability to place them back later
Numpy way to temporarily remove NaN values from array, with ability to place them back later

Time:08-11

I am trying to get a version of an array without the NaN values, with the ability to place them back later. Example:

Array = [1,2,nan,4,5,2,5,6,nan,1,nan,nan,nan,nan,8,7,5,2]
Array_non_nan = [1,2,4,5,2,5,6,1,8,7,5,2]

This can be achieved with array[~np.isnan(array)]. However now I want to be able to remember the locations of the NaN values, and place them back later.

I tried:

array = np.array([nan,  1.,  2.,  3.,  4.,  5., nan,  6.,  7.,  8.,  9.])  
no_nan= array[~np.isnan(array)]      
locs = (np.argwhere(np.isnan(array))).flatten()      
original_array = np.insert(no_nan, obj = locs, values = np.NaN)

However in that case the insertion is unsuccesfull because of a problem with the indexing:

array([nan,  1.,  2.,  3.,  4.,  5.,  6., nan,  7.,  8.,  9.])

I need a vectorized solution, simply writing a loop would be to slow.

CodePudding user response:

You can save the indices where the nan values existed via as a boolean mask. Then use that mask to fill in the the value you want.

a = np.array([nan,  1.,  2.,  3.,  4.,  5., nan,  6.,  7.,  8.,  9.])  
nan_mask = np.isnan(a)
a_no_nan = a[~nan_mask]

So far it is basically what you have. If you want to push the values of a_no_nan to a new array with same shape as a, but keep the nan locations, you can create an empty array, fill it with nan values, then push values using the mask.

c = np.empty_like(a)
c.fill(np.nan)
c[~nan_mask] = a_no_nan
c
# returns:
array([nan,  1.,  2.,  3.,  4.,  5., nan,  6.,  7.,  8.,  9.])  

CodePudding user response:

Well here is the solution. You store the ids and you can insert it back when needed. However I would higly reccomend the other options the other users are suggesting. Store it somewhere or deepcopy

Array = [1,2,np.nan,4,5,2,5,6,np.nan,1,np.nan,np.nan,np.nan,np.nan,8,7,5,2]
Array_non_nan = [x for x in Array if not np.isnan(x)]
list_id = [id_ for id_,elem_ in enumerate(Array) if np.isnan(elem_) ]
for id_ in list_id:
    Array_non_nan.insert(id_, np.nan)
  • Related