I'm trying to add a sum total to an already existing pandas dataframe. The problem is that when I do this my multiindices become just normal indices. How can I prevent that?
df = pd.DataFrame({'date':['2021-06','2021-06','2021-09','2021-08','2021-09'],'type':['t1','t1','t1','t2','t2'], 'other_col':['a','b','b','a','c']})
df1 = df.pivot_table(index='type', columns='date', values='other_col', aggfunc='count').fillna(0)
df1.index = pd.MultiIndex.from_arrays([df1.index, ['count']*len(df1)])
df2 = np.round(df1 / df1.sum(axis=0) * 100, 0).astype(int)
df2.index = pd.MultiIndex.from_arrays([df2.index.get_level_values(0), ['perc']*len(df2)])
df3 = pd.concat([df1, df2]).sort_index()
df3 = df3.assign(**{'row tot': np.where(df3.index.isin([('t1', 'count'), ('t2', 'count')]), df3.sum(axis=1).astype(int), df3.mean(axis=1).astype(int))})
df3 = df3.assign(**{f"{col}":np.where(df3.index.isin([('t1', 'perc'), ('t2', 'perc')]), df3[col].astype(str) '%', df3[col]) for col in df3.columns.tolist()})
column_total = df3[df3.index.isin([('t1', 'count'), ('t2', 'count')])].sum(axis=0).astype(int).to_frame().T
column_total.index = pd.MultiIndex.from_arrays([['col tot'], ])
neither this
df3 = pd.concat([df3, column_total]).sort_index()
nor this works
df3.append(column_total)
Maybe there is also an easier way to add a sum?
CodePudding user response:
You're not really creating a MultiIndex dataframe with column_total
. You need to add a second level so that the dataframes align:
column_total.index = pd.MultiIndex.from_arrays([['col tot'], ['']])
df3 = pd.concat([df3, column_total])
print(df3)
date 2021-06 2021-08 2021-09 row tot
type
t1 count 2.0 0.0 1.0 3
perc 100.0% 0.0% 50.0% 50%
t2 count 0.0 1.0 1.0 2
perc 0.0% 100.0% 50.0% 50%
col tot 2 1 2 5