Replace pandas containing empty list with None-CodePudding

How can I replace all empty list [] values to None in the following df:

import pandas as pd

d = pd.Timestamp.now()

df = pd.DataFrame({'col1': ['a', 'b', 'c'], 
              'col2': [1, list(), 3],
              'col3': ['x', 'y', list()],
              'col4': [d, d, d]})

The solution: df[df.applymap(lambda x: x == [])] = None gives TypeError: Cannot do inplace boolean setting on mixed-types with a non np.nan value

CodePudding user response：

This is tricky! Because of the way pandas (and numpy) coerce lists, vectorizing this may actually be impossible. However, I might be wrong.

A clean, simple solution would be to use applymap to check if each value is [], and then replace using that mask:

df[df.applymap(lambda x: x == [])] = None

Output:

>>> df
  col1  col2  col3
0    a     1     x
1    b  None     y
2    c     3  None

CodePudding user response：

Try where

df.where(df.astype(bool),None,inplace=True)
df
Out[419]: 
  col1  col2  col3
0    a     1     x
1    b  None     y
2    c     3  None

CodePudding user response：

Solved this by using applymap with boolean mask with:

import numpy as np
from collections.abc import Iterable

df[df.applymap(lambda x: len(x) == 0 if isinstance(x, Iterable) else False)] = np.nan