Check if column pair in one dataframe exists in another?-CodePudding

d1 = {'id': ['a','b','c'], 'ref': ['apple','orange','banana']}
df1 = pd.DataFrame(d1)

d2 = {'id': ['a','b','d'], 'ref': ['apple','orange','banana']}
df2 = pd.DataFrame(d2)

I want to see if the column pair of id and ref in df1 exists in df2. I'd like to create a boolean column in df2 to accomplish this.

Desired Output:

d3 = {'id': ['a','b','d'], 'ref': ['apple','orange','banana'], 'check':[True,True,False]}
df2 = pd.DataFrame(d3)

I've tried the following along with a simple assign/isin

df2['check'] = df2[['id','ref']].isin(df1[['id','ref']].values.ravel()).any(axis=1)

df2['check'] = df2.apply(lambda x: x.isin(df1.stack())).any(axis=1)

How can I do this WITHOUT a merge?

CodePudding user response：

I'm not sure why you don't like merge, but you can use isin with tuple:

df2['check'] = df2[['id','ref']].apply(tuple, axis=1)\
                  .isin(df1[['id','ref']].apply(tuple, axis=1))

Output:

  id     ref  check
0  a   apple   True
1  b  orange   True
2  d  banana  False

CodePudding user response：

I think this is what you're looking for:

d1 = {'id': ['a','b','c'], 'ref': ['apple','orange','banana']}
df1 = pd.DataFrame(d1)

d2 = {'id': ['a','b','d'], 'ref': ['apple','orange','banana']}
df2 = pd.DataFrame(d2)

result =  df1.loc[df1.id.isin(df2.id) & df2.ref.isin(df2.ref)]

although a merge would almost certainly be more efficient:

#create a compound key with id   ref
df1["key"] = df1.apply(lambda row: f'{row["id"]}_{row["ref"]}', axis=1)
df2["key"] = df2.apply(lambda row: f'{row["id"]}_{row["ref"]}', axis=1)
#merge df2 on df1 on compound key
df3 =  df1.merge(df2, on="key")
#locate the matched keys in df1
result = df1.set_index("id").loc[df3.id_x]