How to Find Duplicate Element in Python List-CodePudding

How can I find the index of the duplicate element in the array?

mylist = [1,2,3,4,5,6,7,8,9,10,1,2]

How can I find the index number of 1 and 2 repeated array elements here? So, not the first duplicate element, I want to find and delete the trailing duplicate element from the array. 1 2 values at the end. Thanks for your responses.

CodePudding user response：

Single pass remove duplicates:

mylist = [1,2,3,4,5,6,7,8,9,10,1,2]

def remove_duplicates(l):
    seen = {}
    res = []
    for item in l:
        if item not in seen:
            seen[item] = 1
            res.append(item)
    return res

print(remove_duplicates(mylist))

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Which also preserves order:

mylist = [1,10,3,4,5,6,7,8,9,2,1,2]
print(remove_duplicates(mylist))
[1, 10, 3, 4, 5, 6, 7, 8, 9, 2]

CodePudding user response：

The simplest way to do this (unless you want to log where the duplicates are) is to convert the list to a set.

A set can contain only one instance of each value.

You can do this very easily as shown below.

Note: A set is respresented with curly braces { like a dictionary, but does not have key,value pairs.

mylist = [1,2,3,4,5,6,7,8,9,10,1,2]
myset = set(mylist)
// myset = {1,2,3,4,5,6,7,8,9,10}

EDIT: If the order is important, this method will not work as a set does not store the order.

CodePudding user response：

You can find duplicates in a cycle, memorizing previously seen values in a set.

mylist = [1,2,3,4,5,6,7,8,9,10,1,2]

myset = set()
indices_of_duplicates = []

for ind, val in enumerate(mylist):
    if val in myset:
        indices_of_duplicates.append(ind)
    else:
        myset.add(val)

Then you can delete duplicate elements with

for ind in reversed(indices_of_duplicates):
    del mylist[ind]

Note that we are deleting elements starting from the end of the list because otherwise we would shift elements and we would have to update the indices we have found.

CodePudding user response：

If you have to remove duplicates you can use Set()

mylist = [x for x in set([1,2,3,4,5,6,7,8,9,10,1,2])]