Remove NaN values from Pandas DataFrame-CodePudding

I have a sample DataFrame similar to the one below

            a            b           c
4   58.254690  2475.247525  131.665569
6  -58.709564 -2597.402597 -143.492610
7         NaN  2314.814815  145.539223
8  -60.786578 -2032.520325 -154.822728
9  -57.780089 -2283.105023 -140.646976

Is it possible to remove only the NaN values or move it to the bottom of the DataFrame like the following one?

            a            b           c
4   58.254690  2475.247525  131.665569
6  -58.709564 -2597.402597 -143.492610
7  -60.786578  2314.814815  145.539223
8  -57.780089 -2032.520325 -154.822728
9         NaN -2283.105023 -140.646976

CodePudding user response：

Use justify function:

df = pd.DataFrame(justify(df.to_numpy(), invalid_val=np.nan, axis=0, side='up'), 
                  columns=df.columns)
print (df)
           a            b           c
0  58.254690  2475.247525  131.665569
1 -58.709564 -2597.402597 -143.492610
2 -60.786578  2314.814815  145.539223
3 -57.780089 -2032.520325 -154.822728
4        NaN -2283.105023 -140.646970

Or use Series.sort_values per columns with key parameter:

df = df.apply(lambda x: x.sort_values(key = lambda x: x.isna()).tolist())
print (df)

           a            b           c
4  58.254690  2475.247525  131.665569
6 -58.709564 -2597.402597 -143.492610
7 -60.786578  2314.814815  145.539223
8 -57.780089 -2032.520325 -154.822728
9        NaN -2283.105023 -140.646970

CodePudding user response：

remove only (replace):

some_value = 'ItWasNA'

df.fillna(some_value, inplace=True)

move:

for col in df.columns:
    df[col] = df[col].dropna().reset_index(drop=True)