Filter list of dict in python-CodePudding

filter the dictionary based on filter criteria

records = [
    {"category": "automobile", "type": "car", "model": "suv", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "all", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "sedan", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "hatchback", "year": "2010"},
]

filter_value = {
    "model": ["suv"]
}

filter_record = [
    {"category": "automobile", "type": "car", "model": "suv", "year": "2010"},
]

records: records is the list of unfiltered data filter_value : filter_value is the search criteria filter_record: it is the final data after filter, if the filter value is not present in the records, then need to check if there are any records that has 'all' as value for example :-

records = [
    {"category": "automobile", "type": "car", "model": "suv", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "all", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "sedan", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "hatchback", "year": "2010"},
]

filter_value = {
    "model": ["truck"]
}

filter_record = [
    {"category": "automobile", "type": "car", "model": "all", "year": "2010"},
]

in the above example 'truck' is not present in any of the record, so we need to bring out model = 'all'

my code:-

filter_value = {
    "model": ["suv"]
}

filter_record = list(filter(lambda item: (item["model"] in filter_value.get("model") or item["model"] == "all"), records))
print(filter_record)

output:-

[{'category': 'automobile', 'type': 'car', 'model': 'suv', 'year': '2010'}, {'category': 'automobile', 'type': 'car', 'model': 'all', 'year': '2010'}]

can someone help me what I'm missing here, I'm new to python

the expected output should be:-

[{'category': 'automobile', 'type': 'car', 'model': 'suv', 'year': '2010'}]

UPDATED:

new example-

records = [
    {"category": "automobile", "type": "car", "model": "suv", "year": "2019"},
    {"category": "aircraft", "type": "fixed wing", "model": "F35", "year": "2019"},
    {"category": "aircraft", "type": "rotary wing", "model": "blackhawk", "year": "2019"},
    {"category": "automobile", "type": "car", "model": "hatchback", "year": "2010"},
    {"category": "aircraft", "type": "fixed wing", "model": "ALL", "year": "2019"},
    {"category": "aircraft", "type": "fixed wing", "model": "F35", "year": "ALL"},
    {"category": "aircraft", "type": "rotary wing", "model": "blackhawk", "year": "2022"},
    {"category": "aircraft", "type": "rotary wing", "model": "blackhawk", "year": "ALL"},
    {"category": "aircraft", "type": "rotary wing", "model": "ALL", "year": "2022"},
    {"category": "aircraft", "type": "airline", "model": "747", "year": "2019"},
    {"category": "aircraft", "type": "airline", "model": "747", "year": "2022"},
    {"category": "aircraft", "type": "airline", "model": "747", "year": "2030"},
    {"category": "aircraft", "type": "helicopters", "model": "hingeless", "year": "2019"},
    {"category": "aircraft", "type": "helicopters", "model": "teetering", "year": "2019"},
    {"category": "aircraft", "type": "helicopters", "model": "ALL", "year": "ALL"}
    ]

filter_value:

filter_value= {
    "category" : ["aircraft"],
    "model": ["F35","blackhawk","suv","747","sedan"], 
    "year": ["2019","2022","2025"]
    }

expected output:

filter_record = [
    {"category": "aircraft", "type": "fixed wing", "model": "F35", "year": "2019"}, # search combination = "category" : ["aircraft"],  "model": ["F35"], "year": ["2019"]
    {"category": "aircraft", "type": "fixed wing", "model": "F35", "year": "ALL"}, # search combination = "category" : ["aircraft"],  "model": ["F35"], "year": ["2022"], we have record with "caregory" = "aircraft" and "model" = "F35" but not year = 2022, at this time we need to check if we have record with "category" = "aircraft"  "model" = "F35" "year"= "ALL"
    {"category": "aircraft", "type": "fixed wing", "model": "F35", "year": "ALL"}, # search combination = "category" : ["aircraft"],  "model": ["F35"], "year": ["2025"], we have record with "caregory" = "aircraft" and "model" = "F35" but not year = 2025, at this time we need to check if we have record with "category" = "aircraft"  "model" = "F35" "year"= "ALL"
    {"category": "aircraft", "type": "rotary wing", "model": "blackhawk", "year": "2019"}, # search combination = "category" : ["aircraft"],  "model": ["blackhawk"], "year": ["2019"]
    {"category": "aircraft", "type": "rotary wing", "model": "blackhawk", "year": "2022"},# search combination = "category" : ["aircraft"],  "model": ["blackhawk"], "year": ["2022"]
    {"category": "aircraft", "type": "rotary wing", "model": "blackhawk", "year": "ALL"},# search combination = "category" : ["aircraft"],  "model": ["blackhawk"], "year": ["2025"], we have record with "caregory" = "aircraft" and "model" = "blackhawk" but not year = 2025, at this time we need to check if we have record with "category" = "aircraft"  "model" = "blackhawk" "year"= "ALL"
    {"category": "aircraft", "type": "fixed wing", "model": "ALL", "year": "2019"},# search combination = "category" : ["aircraft"],  "model": ["suv"], "year": ["2019"], we have record with "caregory" = 'aircraft' and "year" ='2019' but not "model" = "suv", at this time we need to check if we have record with "category" ="aircraft",  "model" = "ALL", "year" = "2019"
    {"category": "aircraft", "type": "rotary wing", "model": "ALL", "year": "2022"}, # search combination = "category" : ["aircraft"],  "model": ["suv"], "year": ["2022"], we have record with "caregory" = 'aircraft' and "year" ='2022' but not "model" = "suv", at this time we need to check if we have record with "category" ="aircraft",  "model" = "ALL", "year" = "2022"
    {"category": "aircraft", "type": "helicopters", "model": "ALL", "year": "ALL"}, # search combination = "category" : ["aircraft"],  "model": ["suv"], "year": ["2025"], we have record with "caregory" = 'aircraft' but not "model" = "suv" nor and "year" ='2025', at this time we need to check if we have record with "category" ="aircraft",  "model" = "ALL", "year" = "ALL"
    {"category": "aircraft", "type": "airline", "model": "747", "year": "2019"},# search combination = "category" : ["aircraft"],  "model": ["747"], "year": ["2019"]
    {"category": "aircraft", "type": "airline", "model": "747", "year": "2022"}, # search combination = "category" : ["aircraft"],  "model": ["747"], "year": ["2022"]
    {}, # search combination = "category" : ["aircraft"],  "model": ["747"], "year": ["2025"], we have record with "caregory" = 'aircraft' and model = "747" but we do not have year = '2025' or year = 'ALL'
    {"category": "aircraft", "type": "fixed wing", "model": "ALL", "year": "2019"}, # search combination = "category" : ["aircraft"],  "model": ["sedan"], "year": ["2019"], we have record with "caregory" = 'aircraft' and "year" ='2019' but not "model" = "sedan", at this time we need to check if we have record with "category" ="aircraft",  "model" = "ALL", "year" = "2019"
    {"category": "aircraft", "type": "rotary wing", "model": "ALL", "year": "2022"}, # search combination = "category" : ["aircraft"],  "model": ["sedan"], "year": ["2022"], we have record with "caregory" = 'aircraft' and "year" ='2022' but not "model" = "sedan", at this time we need to check if we have record with "category" ="aircraft",  "model" = "ALL", "year" = "2022"
    {"category": "aircraft", "type": "helicopters", "model": "ALL", "year": "ALL"} # search combination = "category" : ["aircraft"],  "model": ["sedan"], "year": ["2025"], we have record with "caregory" = 'aircraft' but not "model" = "sedan" nor and "year" ='2025', at this time we need to check if we have record with "category" ="aircraft",  "model" = "ALL", "year" = "ALL"
]

CodePudding user response：

records = [
    {"category": "automobile", "type": "car", "model": "suv", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "all", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "sedan", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "hatchback", "year": "2010"},
]

filter_value = {
    "model": ["suv", "sedan", "truck"]
}

out = []
for f in filter_value['model']:
    filter_record = list(filter(lambda item: item["model"] == f, records))
    if not filter_record:
        filter_record = list(filter(lambda item: item["model"] == "all", records))
    out.extend(filter_record)
print(out)

[{'category': 'automobile', 'type': 'car', 'model': 'suv', 'year': '2010'}, {'category': 'automobile', 'type': 'car', 'model': 'sedan', 'year': '2010'}, {'category': 'automobile', 'type': 'car', 'model': 'all', 'year': '2010'}]

CodePudding user response：

If you have multiple search terms, an approach like this might work. It's computationally inelegant, but gives you fine control over what you're searching for:

# Records is always a list of dictionaries. Every dictionary has exactly the same
# keys as the others. 

# Any record that has a key whose value is "all", will be returned for any search
# that includes that field. For instance, below, any search for any particular
# model will always return the second record - because whatever "model" you search
# for, "all" meets that description. 

records = [
    {"category": "automobile", "type": "car", "model": "suv", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "all", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "sedan", "year": "2010"},
    {"category": "automobile", "type": "car", "model": "hatchback", "year": "2010"},
    ]


# search_terms is a dictionary where every key must be one of the keys of one of the
# dictionaries comprising records above. Every value is a list of strings. 

search_terms = {
    "model": ["suv", "sedan"]
    }


def search_records(records, search_terms):
    """
    Returns a list of dictionaries, each dictionary being a single record from 
    "records" above.
    """
    filter_record = [] # Initiate an empty list to hold the returned records.
    # step through every key in "search_terms"
    for search_field, terms in search_terms.items(): 
        # within that key, step through every specific value searched for
        for term in terms:
            # search through every record in records
            for record in records:
                if all([record not in filter_record, (record[search_field] == term or record[search_field] == 'all')]):
                    filter_record.append(record)
                    
    return filter_record

filter_record = search_records(records, search_terms)

for record in filter_record:
    print (record)

Output:

{'category': 'automobile', 'type': 'car', 'model': 'suv', 'year': '2010'}
{'category': 'automobile', 'type': 'car', 'model': 'all', 'year': '2010'}
{'category': 'automobile', 'type': 'car', 'model': 'sedan', 'year': '2010'}

To validate that it works with different types of records and multiple search terms, change your records to this:

records = [
    {"category": "automobile", "type": "car", "model": "suv", "year": "2010"},
    {"category": "aircraft", "type": "fixed wing", "model": "F35", "year": "2019"},
    {"category": "aircraft", "type": "rotary wing", "model": "blackhawk", "year": "1999"},
    {"category": "automobile", "type": "car", "model": "hatchback", "year": "2010"},
    ]

and your search terms to:

search_terms = {
    "model": ["suv"],
    "category" : ["aircraft"]
    }

and calling the same function you get this output:

{'category': 'automobile', 'type': 'car', 'model': 'suv', 'year': '2010'}
{'category': 'aircraft', 'type': 'fixed wing', 'model': 'F35', 'year': '2019'}
{'category': 'aircraft', 'type': 'rotary wing', 'model': 'blackhawk', 'year': '1999'}

Similarly changing the search terms to

search_terms = {
    "type" : ["fixed wing"], 
    "model" : ["hatchback"]
    }

gives you:

{'category': 'aircraft', 'type': 'fixed wing', 'model': 'F35', 'year': '2019'}
{'category': 'automobile', 'type': 'car', 'model': 'hatchback', 'year': '2010'}

Does that do what you're looking for?