I need to remove nan values from python list. The problem is that the list can contain lists of one element, lists of two elements or nan values. E.g.:

import numpy as np

mylist = [[1,2],float('nan'),[8],[6],float('nan')]
print(mylist)
newlist = [x for x in mylist if not np.isnan(x)]
print(newlist)

Unfortunately calling np.isnan() on list with more than 1 element gives error:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The same goes for pandas.isnull

math.isnan() on the other hand gives:

TypeError: must be real number, not list

Is there a short way to solve this problem?

CodePudding user response：

The issue is that you cannot test the NaN status on a list. You should only test is on floats.

For this you can use:

newlist = [x for x in mylist if not isinstance(x, float) or not np.isnan(x)]

alternatively:

newlist = [x for x in mylist if isinstance(x, list) or not np.isnan(x)]

Due to short-circuiting, the np.isnan(x) will only be evaluated if x is a float (or not a list in the alternative), which won't trigger an error.

output: [[1, 2], [8], [6]]