Functional approach to group DataFrame columns into MultiIndex-CodePudding

Is there a simpler functional way to group columns into a MultiIndex?

# Setup
l = [...]
l2,l3,l4 = do_things(l, [2,3,4])
d = {2:l2, 3:l3, 4:l4}
# Or,
l = l2 = l3 = l4 = list(range(20))

Problems with my approaches:

# Cons:
# * Complicated
# * Requires multiple iterations over the dictionary to occur
#   in the same order. This is guaranteed as the dictionary is
#   unchanged but I'm not happy with the implicit dependency.
df = pd.DataFrame\
    ( zip(*d.values())
    , index=l
    , columns=pd.MultiIndex.from_product([["group"], d.keys()])
    ).rename_axis("x").reset_index().reset_index()

# Cons:
# * Complicated
# * Multiple assignments
df = pd.DataFrame(d, index=l).rename_axis("x")
df.columns = pd.MultiIndex.from_product([["group"],df.columns])
df = df.reset_index().reset_index()

I'm looking for something like:

df =\
    ( pd.DataFrame(d, index=l)
    . rename_axis("x")
    . group_columns("group")
    . reset_index().reset_index()
    )

Result:

   index  x group
                2  3  4
0      0  2     0  0  0
1      1  2     0  0  0
2      2  2     0  0  0
3      3  2     0  0  0
4      4  1     0  0  0
5      5  2     0  0  0
6      6  1     0  0  0
7      7  2     0  0  0
8      8  4     0  1  1
9      9  4     0  1  1
10    10  4     0  1  1
11    11  0     0  1  1
12    12  1     0  1  1
13    13  1     0  1  1
14    14  3     1  2  2
15    15  1     1  2  2
16    16  1     1  2  3
17    17  1     1  2  3
18    18  4     1  2  3
19    19  3     1  2  3
20    20  4     1  2  3
21    21  4     1  2  3
22    22  4     1  2  3
23    23  4     1  2  3

CodePudding user response：

It is probably easiest just to reformat the dictionary and pass it to the DataFrame constructor:

# Sample Data
size = 5
lst = np.arange(size)   10
d = {2: lst, 3: lst   size, 4: lst   (size * 2)}

df = pd.DataFrame(
    # Add group level by changing keys to tuples
    {('group', k): v for k, v in d.items()},
    index=lst
)

Output:

   group        
       2   3   4
10    10  15  20
11    11  16  21
12    12  17  22
13    13  18  23
14    14  19  24

Notice that tuples get interpreted as a MultiIndex automatically

This can be followed with whatever chain of operations desired:

df = pd.DataFrame(
    {('group', k): v for k, v in d.items()},
    index=lst
).rename_axis('x').reset_index().reset_index()

df:

  index   x group        
                2   3   4
0     0  10    10  15  20
1     1  11    11  16  21
2     2  12    12  17  22
3     3  13    13  18  23
4     4  14    14  19  24

It is also possible to combine steps and generate the complete DataFrame directly:

df = pd.DataFrame({
    ('index', ''): pd.RangeIndex(len(lst)),
    ('x', ''): lst,
    **{('group', k): v for k, v in d.items()}
})

df:

  index   x group        
                2   3   4
0     0  10    10  15  20
1     1  11    11  16  21
2     2  12    12  17  22
3     3  13    13  18  23
4     4  14    14  19  24

Naturally any combination of dictionary comprehension and pandas operations can be used.