Use numpy to mask a row containing only zeros-CodePudding

I have a large array of point cloud data which is generated using the azure kinect. All erroneous measurements are assigned the coordinate [0,0,0]. I want to remove all coordinates with the value [0,0,0]. Since my array is rater large (1 million points) and since U need to do this process in real-time, speed is of the essence.

In my current approach I try to use numpy to mask out all rows that contain three zeroes ([0,0,0]). However, the np.ma.masked_equal function does not evaluate an entire row, but only evaluates single elements. As a result, rows that contain at least one 0 are already filtered by this approach. I only want rows to be filtered when all values in the row are 0. Find an example of my code below:

my_data = np.array([[1,2,3],[0,0,0],[3,4,5],[2,5,7],[0,0,1]])

my_data = np.ma.masked_equal(my_data, [0,0,0])

my_data = np.ma.compress_rows(my_data)

output

array([[1, 2, 3],
       [3, 4, 5],
       [2, 5, 7]])

desired output

array([[1, 2, 3],
       [3, 4, 5],
       [2, 5, 7],
       [0, 0, 1]])`

CodePudding user response：

Find all data points that are 0 (doesn't require np.ma module) and then select all rows that do not contain all zeros:

import numpy as np
my_data = np.array([[1, 2, 3], [0, 0, 0], [3, 4, 5], [2, 5, 7], [0, 0, 1]])

my_data[~(my_data == 0).all(axis= 1)]

Output:

array([[1, 2, 3],
       [3, 4, 5],
       [2, 5, 7],
       [0, 0, 1]])

CodePudding user response：

Instead of using the np.ma.masked_equal and np.ma.compress_rows functions, you can use the np.all function to check if all values in a row are equal to [0, 0, 0]. This should be faster than your method as it evaluates all values in a row at once.

mask = np.all(my_data == [0, 0, 0], axis=1)
my_data = my_data[~mask]