Extract Unique value in nested lists-CodePudding

I have a list with different and repeated IP Addresses. So I want to extract a unique IP Address from the nested list.

import re
pattern='(\d .{13,15})'
matches =[re.findall(pattern,str(x)) for x in dfs['Task Category']]
print(matches)  

import numpy as np 
new_list = list(np.concatenate(matches)) 
print(new_list)

[['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.178.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], [], [], [], [], [], ['192.168.101.115.'], ['192.168.101.178.'], [], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.82.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.79.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.79.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.79.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.43.'], ['192.168.101.62.'], [], ['192.168.101.62.'], [], ['192.168.101.62.'], ['192.168.101.115.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.43.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.82.'], ['192.168.101.91.'], [], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.115.'], ['192.168.101.115.'], ['192.168.101.178.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.82.'], ['192.168.101.91.'], ['192.168.101.82.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.82.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.82.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.62.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.115.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.62.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.91.'], ['192.168.101.252.'], ['192.168.101.252.'], ['192.168.101.252.'], ['192.168.101.252.'], ['192.168.101.252.'], ['192.168.101.252.']

CodePudding user response：

Use a set or a Counter:

set(match for x in dfs['Task Category'] for match in re.findall(pattern,str(x))

Or:

from collections import Counter

Counter(match for x in dfs['Task Category'] for match in re.findall(pattern,str(x))

Or, using pandas:

dfs['Task Category'].str.extractall(f'({pattern})')[0].unique()

CodePudding user response：

It is hard to tell, but I guess your list is just a list of a list, so:

my_list = [[1,2,3], [1,2,3]] # This is my guessing of your list

unique_list = []
  
# traverse for all elements
for x in my_list:
        # check if exists in unique_list or not
        if x not in unique_list:
            unique_list.append(x)

The Output:

unique_list
[[1, 2, 3]] # If you need the list inside, just: unique_list[0]