How to combine two lists while removing duplicates item from dynamic input in python-CodePudding

I have a list like this:

[{'groups': ['Engineer', 'It', 'Office Floor 2', 'Penang']}, {'groups': ['Engineer', 'Penang', 'Testnew']}]

All I wanted to do is separate it to delete duplicated item in the list, for example:

['Engineer', 'It', 'Office Floor 2', 'Penang', 'Testnew']

I tried with this code:

list_1 = ['Engineer', 'It', 'Office Floor 2', 'Penang']
list_2 = ['Engineer', 'Penang', 'Testnew']

set_1 = set(list_1)
set_2 = set(list_2)

list_2_items_not_in_list_1 = list(set_2 - set_1)
combined_list = list_1   list_2_items_not_in_list_1

print(combined_list)

Although it worked but how can I modify it to a dynamic list? Since my list is not static and it may change according to how many user input.

CodePudding user response：

If the orders of appearance is a constraint, you could do:

lst = [{'groups': ['Engineer', 'It', 'Office Floor 2', 'Penang']}, {'groups': ['Engineer', 'Penang', 'Testnew']}]

seen = set()
result = []
for d in lst:
    for e in d["groups"]:
        if e not in seen:
            result.append(e)
            seen.add(e)
print(result)

Output

['Engineer', 'It', 'Office Floor 2', 'Penang', 'Testnew']

If the order of appearance is not a constraint:

from itertools import chain
result = list(set(chain.from_iterable(d["groups"] for d in lst)))
print(result)

Output

['Office Floor 2', 'Penang', 'Testnew', 'It', 'Engineer']

Or simply:

result = list(set(e for d in lst for e in d["groups"]))

CodePudding user response：

It should be like this -

set_1 = list(set(list_1   list2))

CodePudding user response：

Or you can try this:

from itertools import chain
lst = [{'groups': ['Engineer', 'It', 'Office Floor 2', 'Penang']}, {'groups': ['Engineer', 'Penang', 'Testnew']}]
list(set(chain.from_iterable(map(lambda d: d['groups'], lst))))