Find unique words in a list of lists in python-CodePudding

I have a list of lists that I would like to iterate over using a for loop, and create a new list with only the unique words. This is similar to a question asked previously, but I could not get the solution to work for me for a list within a list

For example, the nested list is as follows:

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']],

The desired output would be a single list:

List_Unique = [['is','and','so','he','his','run']]

I have tried the following two variations of code, but the output of all of them is a list of repeats:

unique_redundant = [] 
for i in redundant_search:
    redundant_i = [j for j in i if not i in unique_redundant]
    unique_redundant.append(redundant_i)
unique_redundant


unique_redundant = [] 
for list in redundant_search:
    for j in list:
        redundant_j = [i for i in j if not i in unique_redundant]
    unique_redundant.append(length_j)
unique_redundant

Example output given for the above two (incorrect) variations

(I ran the code on my real set of data and it gave repeating lists within lists of the same pair of words, though this isn't the actual two words, just an example):

List_Unique = [['is','and'],['is','and'],['is','and']]

CodePudding user response：

First flatten the list with itertools.chain, then use set to return the unique elements and pass that into a list:

from itertools import chain

if __name__ == '__main__':
    print([{list(chain(*list_of_lists))}])

CodePudding user response：

Use itertools.chain to flatten the list and dict.fromkeys to keep the unique values in order:

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']],

from itertools import chain
List_Unique = [list(dict.fromkeys(chain.from_iterable(*ListofList)))]

CodePudding user response：

Just index out nested list with the help of while and acquire all the values in new list while cnt<len(listoflist)

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]
list_new=[]
cnt=0
while cnt<len(ListofList):
    for i in ListofList[cnt]:
        if i in list_new:
            continue
        else:
            list_new.append(i)
    cnt =1
print(list_new)

OUTPUT

['is', 'and', 'so', 'he', 'his', 'run']

CodePudding user response：

flat_list = [item for sublist in ListofList for item in sublist]

# use this if order should not change
List_Unique = []
for item in flat_list:
    if item not in List_Unique:
        List_Unique.append(item)


# use this if order is not an issue
# List_Unique = list(set(flat_list))

CodePudding user response：

You could try this:

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]
uniqueItems = []
for firstList in ListofList:
    for item in firstList:
        if item not in uniqueItems:
            uniqueItems.append(item)
print(uniqueItems)

It uses a nested for loop to access each item and check whether it is in uniqueItems.

CodePudding user response：

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]
list_new=[]
cnt=0
while cnt<len(ListofList):
    for i in ListofList[cnt]:
        if i in list_new:
            continue
        else:
            list_new.append(i)
    cnt =1
print(list(list_new))

O/P

[['is','and','so','he','his','run']]