Pandas discard items in set using a different set-CodePudding

I have two columns in a pandas dataframe; parents and cte. Both columns are made up of sets. I want to use the cte column to discard overlapping items in the parents column. The dataframe is made up of over 6K rows. Some of the cte rows have empty sets.

Below is a sample:

data = {'parents': [{'loan_agreement', 'select', 'opportunity', 'sales.dim_date', 'sales.flat_opportunity_detail', 'dets', 'dets2', 'channel_partner'}
,{'seed', 'dw_salesforce.sf_dw_partner_application'}], 
        'cte': [{'dets', 'dets2'}, {'seed'}]}
df = pd.DataFrame(data)
df

I've used .discard(cte) previously but I can't figure out how to get it to work.

I would like the output to look like the following:

data = {'parents': [{'loan_agreement', 'select', 'opportunity', 'sales.dim_date', 'sales.flat_opportunity_detail', 'channel_partner'}
,{'dw_salesforce.sf_dw_partner_application'}], 
        'cte': [{'dets', 'dets2'}, {'seed'}]}
df = pd.DataFrame(data)
df

NOTE: dets, dets2 and seed have been removed from the corresponding parents cell.

Once the cte is compared to the parents, I don't need data from that row again. The next row will only compare data on that row and so on.

CodePudding user response：

You need to use a loop here.

A list comprehension will likely be the fastest:

df['parents'] = [P.difference(C) for P,C in zip(df['parents'], df['cte'])]

output:

                                             parents            cte
0  {channel_partner, select, opportunity, loan_ag...  {dets, dets2}
1          {dw_salesforce.sf_dw_partner_application}         {seed}