Shift all NaN values in pandas to the left-CodePudding

I have a (250, 33866) dataframe. As you can see in the picture, all the NaN values are at the end of each row. I would like to shift those NaNvalues ti the left of the dataframe. At the same time I wanna keep the 0 column (which refers to the Id) in its place (stays the first one).

I was trying to define a function that loops over all rows and columns to do that, but figured it will be very inefficient for large data. Any other options? Thanks

CodePudding user response：

You could reverse the columns of df, drop NaNs; build a DataFrame and reverse it back:

out = pd.DataFrame(df.iloc[:,::-1].apply(lambda x: x.dropna().tolist(), axis=1).tolist(), 
                   columns=df.columns[::-1]).iloc[:,::-1]

For example, for a DataFrame that looks like below:

   col0  col1  col2  col3  col4
1   1.0   2.0   3.0  10.0  20.0
2   1.0   2.0   3.0   NaN   NaN
3   1.0   2.0   NaN   NaN   NaN

the above code produces:

   col0  col1  col2  col3  col4
0   1.0   2.0   3.0  10.0  20.0
1   NaN   NaN   1.0   2.0   3.0
2   NaN   NaN   NaN   1.0   2.0