How to replace null values in a column with mean of non null values in a particular column by using-CodePudding

The dataset i have taken is having columns country,coal_production_changepct,gasprodchangepct,year.There are null values in coal prod change pct and gas prod change pct and i want to replace the null values with average of coal prod change pct non null values and gas prod change pct non null values.the dataframe is as looks like below img.

[{"metadata":{"trusted":true},"cell_type":"code","source":"sample_df.loc[490:500,['country','coal_prod_change_pct','year','gas_prod_change_pct']]","execution_count":79,"outputs":[{"output_type":"execute_result","execution_count":79,"data":{"text/plain":"                  country  coal_prod_change_pct  year  gas_prod_change_pct\n490               Ukraine              2.737000  2018             1.463000\n491               Ukraine             -2.299000  2019            -0.481000\n492               Ukraine             -4.111211  2020             1.197368\n493  United Arab Emirates                   NaN  2001             2.553000\n494  United Arab Emirates                   NaN  2002            10.239000\n495  United Arab Emirates                   NaN  2003             3.227000\n496  United Arab Emirates                   NaN  2004             3.349000\n497  United Arab Emirates                   NaN  2005             3.240000\n498  United Arab Emirates                   NaN  2006             2.092000\n499  United Arab Emirates                   NaN  2007             3.074000\n500  United Arab Emirates                   NaN  2008            -0.099000","text/html":"\n\n\n  \n    \n      \n      \n      \n      \n      \n    \n  \n  \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n    \n      \n      \n      \n      \n      \n    \n  \ncountrycoal_prod_change_pctyeargas_prod_change_pct490Ukraine2.73700020181.463000491Ukraine-2.2990002019-0.481000492Ukraine-4.11121120201.197368493United Arab EmiratesNaN20012.553000494United Arab EmiratesNaN200210.239000495United Arab EmiratesNaN20033.227000496United Arab EmiratesNaN20043.349000497United Arab EmiratesNaN20053.240000498United Arab EmiratesNaN20062.092000499United Arab EmiratesNaN20073.074000500United Arab EmiratesNaN2008-0.099000\n"},"metadata":{}}]}]


country_grp = sample_df.groupby('country')

country_grp\['coal_prod_change_pct'\].fillna(country_grp\['coal_prod_change_pct'\].mean())

country_grp\['coal_prod_change_pct'\].apply(lambda x: x.fillna(x.mean()))

but in the second method there is no inplace = true as we apply method

CodePudding user response：

We usually do transform

filler = country_grp['coal_prod_change_pct'].transform('mean')
sample_df['coal_prod_change_pct'].fillna(filler, inplace=True)

CodePudding user response：

You can replace null values in a column with the mean of non-null values in that column by using the fillna() method.

For example:

import pandas as pd df = pd.DataFrame({'A':[1,2,3,4,5], 'B':[1,2,3,4,5]}) df['B'].fillna(df['B'].mean(), inplace=True)