Pandas combine dataframes elementwise (in elegant way)-CodePudding

Is there a simple way to combine 2 dataframes elementwise? It seems that the pd.DataFrame.combine works columnwise only.

In the example below, I have 2 dataframes contaning sets. The goal is to get intersection of sets from 2 dataframes (which is done with & operator).

import pandas as pd
import numpy as np

def combine_elementwise(df, df2, func):
    def f(c,c2):
        return c.combine(c2, func)
    return df.combine(df2, f)

df = pd.DataFrame([
        [{1,2}, {3,4}],
        [{5,6}, {7,8}]
        ])

df2 = pd.DataFrame([
        [{9,2}, {1}],
        [{5,6}, {8,9}]
        ])

# this combines columnwise
print('Columnwise results:')
print(df.combine(df2, lambda c,c2: c&c2)) 

# this combines elementwise but is ugly
print('Elementwise results (what I need):')
print(combine_elementwise(df,df2, lambda a,b: a&b))

Columnwise results:
      0      1
0  True  False
1  True   True

Elementwise results (what I need):
        0    1
0     {2}   {}
1  {5, 6}  {8}

I wrote combine_elementwise function which does what I'm looking for, but it's ugly and I'm wondering if there's a simpler way to accomplish the same.

CodePudding user response：

You could do it like:

combined_df = pd.DataFrame(df.values & df2.values)
print(combined_df)

# output:
#         0    1
# 0     {2}   {}
# 1  {5, 6}  {8}

CodePudding user response：

A little bit logic here

out = df-(df-df2)
Out[242]: 
        0    1
0     {2}   {}
1  {5, 6}  {8}