Combine lists of dictionaries with different values-CodePudding

I have list of dictionaries with matching keys like this.

[{'account_general_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}, 
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}, 
{'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}}, 
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}]

I need to combine all matching keys together with the value inside the dicts where if there is true for some key its given priority over false.

So result should look something like

{'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}}, 
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}]

What would the fastest way to achieve this as speed is crucial to perform this.

My attempt Assuming list is sort based on keys and I have a variable x to know how many times a key repeats in a list


new_list = np.array_split(new_list, len(permission))

for i in new_list:
    for j in i:
        for k, l in j.items():
            for m, n in l.items():
                if n == True:
                    l[m] = n

This partially works, also doesn't look the cleanest.

CodePudding user response：

data = [{'account_general_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}, 
        {'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}, 
        {'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}}, 
        {'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}]

# Let's make a new dictionary~
new_data = {}
for x in data:
    # Each "dictionary" in your list appears to just be a single key/value pair.
    key, values = tuple(x.items())[0]
    # If we've added this key to the dictionary already...
    if key in new_data:
        for k in values:
            # If that inner key isn't already True, or doesn't exist.
            if not new_data[key].get(k):
                # Overwrite this inner key's value OR add the new key:value pair.
                new_data[key][k] |= values[k]
    # Otherwise we can just add this dict to the new dict.
    else:
        new_data |= x

print(new_data)

# Output:

{'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}, 
 'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}

An overkill, but one-line approached using pandas could look like:

data = (pd.concat([pd.DataFrame.from_dict(x, 'index') for x in data])
          .groupby(level=0)
          .max()
          .to_dict('index'))

print(data)

# Output

{'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}, 
 'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}

CodePudding user response：

this should work

from collections import Counter

data = [{'account_general_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}, 
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}, 
{'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}}, 
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}]

counters = {'account_general_permission': Counter(), 'control_section_permission': Counter()} 

for element in data:
    for key, data in element.items():
        counters[key].update(data)


result = [{key: {item: bool(val) for item, val in data.items()}} for key, data in counters.items()]
result

output:

{'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}}, 
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}

CodePudding user response：

lod = [{'account_general_permission': {'view': False, 'create': False, 'edit':
                                       False, 'delete': False}},
       {'control_section_permission': {'view': False, 'create': False, 'edit':
                                       False, 'delete': False}},
       {'account_general_permission': {'view': True, 'create': True, 'edit':
                                       True, 'delete': False}},
       {'control_section_permission': {'view': False, 'create': False, 'edit':
                                       False, 'delete': False}}]

result = {}
for d in lod:
    perm_name, perms = next(iter(d.items()))
    result[perm_name] = {perm_elem: any([perm_val,
                                         result.get(perm_name, {}
                                                    ).get(perm_elem, False)])
                         for perm_elem, perm_val in perms.items()}

{'account_general_permission': {'create': True,
                                'delete': False,
                                'edit': True,
                                'view': True},
 'control_section_permission': {'create': False,
                                'delete': False,
                                'edit': False,
                                'view': False}}

The result is a dict of dicts, though, but I feel like it's a more natural data structure for this task.

CodePudding user response：

I don't know about the numpy stuff, but your example applied to the original list is already reasonable, especially considering that this is a small data set. Iteration is faster than indexing and a dictionary's update function that is written in C is a bit faster still. So, using a defaultdict to create items as needed, you could do

import collections

data = [{'account_general_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}, 
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}},
{'account_general_permission': {'view': True, 'create': True, 'edit': True, 'delete': False}},
{'control_section_permission': {'view': False, 'create': False, 'edit': False, 'delete': False}}]

resolved = collections.defaultdict(lambda: {'view': False, 'create': False,
    'edit': False, 'delete': False})

for perms in data:
    for perm_name, perm_vals in perms.items():
        resolved[perm_name].update((n,v) for n,v in perm_vals.items() if v)

for k,v in resolved.items():
    print(f"{k}: {v}")

This solution does not update the dictionaries in the original table.

CodePudding user response：

Let's get crazy and functional:

lod = [{'account_general_permission': {'view': False, 'create': False, 'edit':
                                       False, 'delete': False}},
       {'control_section_permission': {'view': False, 'create': False, 'edit':
                                       False, 'delete': False}},
       {'account_general_permission': {'view': True, 'create': True, 'edit':
                                       True, 'delete': False}},
       {'control_section_permission': {'view': False, 'create': False, 'edit':
                                       False, 'delete': False}}]

from functools import reduce
from itertools import groupby


def get_key(d):
    return next(iter(d.keys()))


def get_val(d):
    return next(iter(d.values()))


def merge_dicts(d, perm_d):
    unwrappe_d = get_val(perm_d)
    return {p: any(d.get(p) for d in (d, unwrappe_d))
            for p in unwrappe_d}


grouped = groupby(sorted(lod, key=get_key), get_key)

result = [{p: reduce(merge_dicts, perm_dicts, {})}
          for p, perm_dicts in grouped]

This one also depends on all the "inner keys" ('view', 'edit' etc) present. I'll think on how it could be improved.