python getting the dicts of a list with a particular key/value pair when key:None exists-CodePudding

I would like to extract the dicts of a list of dicts as follows:

mylist= [{'a':1,'b':3},{'a':71,'d':33},{'e':51,'b':63},{'e':51,'b':63}]

under the condition that a dictionary contains a particular key/pair value. Observe that the dictionaries are not all of the same length, i.e. of the same keys

If that would the case this would work:

#given
condition = {'a':1}

key = next(iter(condition.keys())).lower()
value = next(iter(condition.values())).lower()
filtered = [doc for doc in self.data if value.lower() in doc.get(key, "justastring").lower()]

So far so good.

Now imagine that the list of dicts is a API response that might look like this:

mylist= [{'a':None,'b':3},{'a':71,'d':33},{'e':None,'b':63},{'e':51,'b':63}]

The above code gives the error:

TypeError: argument of type 'NoneType' is not iterable

So {'a': None} should behave as if a would not exist.

Some ideas?

CodePudding user response：

It seems like you can just use a list comprehension with a condition that all the keys in condition are in the dict and the value is the same. That might look like:

mylist= [{'a':None,'b':3},{'a':1,'d':33},{'e':None,'b':63},{'a':51,'b':63}]
condition = {'a':1}

[d for d in mylist if all(k in d and d[k] == v for k, v in condition.items())]
# [{'a': 1, 'd': 33}]

This will allow a condition with more than one key and should handle conditions like condition = {'a':None} without issue (it would produce [{'a':None,'b':3}] with the above input).

CodePudding user response：

You can add an extra condition in the list comprehension to skip over any None values.

filtered = [
    doc for doc in self.data
    # First check for None then check for membership in doc
    if doc.get(key) is not None and value.lower() in doc.get(key, "justastring").lower()
]

if will evaluate lazily i.e. it will evaluate to False if the first predicate is False without even checking the second predicate in an and expression. Equally, a boolean or expression will evaluate to True if the first predicate evaluates to True.

For example

In [16]: a = 1

# a == 2 evaluates to False, no need to evaluate the second predicate
In [17]: a == 2 and a.lower() == 1
Out[17]: False

# Similarly a == 1 evaluates to True, no need to evaluate the second predicate
In [18]: a == 1 or a.lower() == 1
Out[18]: True

This way your code will never try to evaluate var in None.

If you want to be even safer you could do the following to guard against non-str values of value. But this might not conform to your API spec.

filtered = [
    doc for doc in self.data
    # First check for string then check for membership in doc
    if isinstance(doc.get(key), str) and value.lower() in doc.get(key, "justastring").lower()
]