How to keep the values with most frequent prefix in a groupby pandas dataframe?-CodePudding

Let's say I have this dataframe :

    Country Market
0   Spain   m1_name
1   Spain   m1_location
2   Spain   m1_size
3   Spain   m2_location
4   USA     m1_name
5   USA     m2_name
6   USA     m3_size
7   USA     m3_location

I want to group on the "Country" columns and to keep the records with the most frequent records in the groupby object. The expected result would be :

    Country Market
0   Spain   m1_name
1   Spain   m1_location
2   Spain   m1_size
6   USA     m3_size
7   USA     m3_location

I already tried extracting the prefix, then getting the mode of the prefix on the dataframe and merging rows with this mode, but I feel that a more direct and more efficient solution exists.

Here is the working sample code below for reproducible results :

df = pd.DataFrame({
    "Country": ["Spain","Spain","Spain","Spain","USA","USA","USA","USA"],
    "City": ["m1_name","m1_location","m1_size","m2_location","m1_name","m2_name","m3_size","m3_location"]                
                    })
df['prefix'] = df['City'].str[1]
modes = df.groupby('Country')['prefix'].agg(pd.Series.mode).rename("modes")
df = df.merge(modes, how="right", left_on=['Country','prefix'], right_on=['Country',"modes"])
df = df.drop(['modes','prefix'], axis = 1)
print(df)

Country         City
0   Spain      m1_name
1   Spain  m1_location
2   Spain      m1_size
3     USA      m3_size
4     USA  m3_location

CodePudding user response：

You can try groupby and apply to filter group rows

out = (df.assign(prefix=df['City'].str.split('_').str[0])
       .groupby('Country')
       .apply(lambda g: g[g['prefix'].isin(g['prefix'].mode())])
       .reset_index(drop=True)
       .drop('prefix',axis=1))

print(out)

  Country         City
0   Spain      m1_name
1   Spain  m1_location
2   Spain      m1_size
3     USA      m3_size
4     USA  m3_location

CodePudding user response：

Use:

In [575]: df['Prefix_count'] = df.groupby(['Country', df.City.str.split('_').str[0]])['City'].transform('size')

In [589]: idx = df.groupby('Country')['Prefix_count'].transform(max) == df['Prefix_count']

In [593]: df[idx].drop('Prefix_count', 1)
Out[593]: 
  Country         City
0   Spain      m1_name
1   Spain  m1_location
2   Spain      m1_size
6     USA      m3_size
7     USA  m3_location