How to fill the nan's at the end of every column with 0's in pandas python?-CodePudding

I want to forward fill my dataframe with a custom value - like 0. But pandas dosent allow to ffill with custom value. It only takes the last available value in every column and fills the nan values at the end with it. So was wondering if there was a better way to do this in python.

df = 

nan  1  2  nan
1    4  5  2
nan  6  7  nan
nan  nan 8 nan
nan  nan 8 nan

Expected Output:

nan  1  2  nan
1    4  5  2
0    6  7  0
0   0  8   0
0   0  8   0

CodePudding user response：

Let us do

df = df.where(df.ffill().isna() | df.notna(),0)
Out[108]: 
     1    2  3    4
0  NaN  1.0  2  NaN
1  1.0  4.0  5  2.0
2  0.0  6.0  7  0.0
3  0.0  0.0  8  0.0
4  0.0  0.0  8  0.0

df.fillna(0).mask(df.ffill().isna())
Out[111]: 
     1    2  3    4
0  NaN  1.0  2  NaN
1  1.0  4.0  5  2.0
2  0.0  6.0  7  0.0
3  0.0  0.0  8  0.0
4  0.0  0.0  8  0.0

CodePudding user response：

You can use boolean indexing:

m1 = df.notna()
m2 = m1.cummax()

out = df.mask(m2&~m1, 0)

output:

     0    1  2    3
0  NaN  1.0  2  NaN
1  1.0  4.0  5  2.0
2  0.0  6.0  7  0.0
3  0.0  0.0  8  0.0
4  0.0  0.0  8  0.0

If you have intermediate NaN and do not want to fill/mask those:

m1 = df.isna()
out = df.mask(m1&m1[::-1].cummin(), 0)

example:

     0    1    2    3
0  NaN  1.0  2.0  NaN
1  1.0  4.0  5.0  2.0
2  0.0  6.0  NaN  0.0
3  0.0  0.0  8.0  0.0
4  0.0  0.0  8.0  0.0