Remove duplicates dictionaries from list of dictionaries in python-CodePudding

I want to remove duplicates dictionaries from list of dictionaries. I am trying to make configurable code to work on any field instead of making field specific.

Input Data

dct = {'Customer_Number': 90617174, 
       'Phone_Number': [{'Phone_Type': 'Mobile', 'Phone': [12177218280.0]}, 
                        {'Phone_Type': 'Mobile', 'Phone': [12177218280.0]}],
       'Email': [{'Email_Type': 'Primary', 
                  'Email': ['[email protected]']},
                 {'Email_Type': 'Primary', 
                  'Email': ['[email protected]']}]
      }

Expected Output:

{'Customer_Number': 90617174, 
                'Email': [{'Email_Type': 'Primary', 
                           'Email': ['[email protected]']}], 
                'Phone_Number': [{'Phone_Type': 'Mobile', 
                                  'Phone': [12177218280]}]}



**Code tried:**

    res_list = []
    for key,value in address_dic.items():
        if isinstance(value,list):
            for i in range(len(value)):
                if value[i] not in value[i   1:]:
                   res_list.append(value[i])
    
    dic.append((res_list))


**Output Getting**

    type: [[{'Email_Type': 'Primary', 'Email': ['[email protected]']}, {'Email_Type': 'alternate', 'Email': ['[email protected]', '[email protected]']}], [], [{'Phone_Type': 'Mobile', 'Phone': [12177218280.0]}, {'Phone_Type': 'work', 'Phone': [nan]}, {'Phone_Type': 'home', 'Phone': [2177218280.0]}], []]

CodePudding user response：

Write a function to dedupe lists:

def dedupe_list(lst):
    result = []
    for el in lst:
        if el not in result:
            result.append(el)
    return result

def dedupe_dict_values(dct):
    result = {}
    for key in dct:
        if type(dct[key]) is list:
            result[key] = dedupe_list(dct[key])
        else:
            result[key] = dct[key]
    return result

Test it:

deduped_dict = {'Customer_Number': 90617174, 
                'Email': [{'Email_Type': 'Primary', 
                           'Email': ['[email protected]']}], 
                'Phone_Number': [{'Phone_Type': 'Mobile', 
                                  'Phone': [12177218280]}]}


dedupe_dict_values(dct) == deduped_dict
## Out[12]: True