Creating cartesian product of sub-dictionary key-val pairs-CodePudding

I have this dictionary :

d = {
    'hosts': [{'hostname': 'ijk,uvw,xyz', 'ip': '127.0.0.3,127.0.0.4,127.0.0.5', 'extra': 'check-me,check-this,check-it'}]}

I want a dictionary with the cartesian product of the key-val pairs inside .

I need to create the below dictionary out of it:

d = {
    'hosts': [
        {'hostname': 'ijk', 'ip': '127.0.0.3', 'extra': 'check-me'},
        {'hostname': 'uvw', 'ip': '127.0.0.4', 'extra': 'check-me'},
        {'hostname': 'xyz', 'ip': '127.0.0.5', 'extra': 'check-me'},
        {'hostname': 'ijk', 'ip': '127.0.0.3', 'extra': 'check-it'},
        {'hostname': 'uvw', 'ip': '127.0.0.4', 'extra': 'check-it'},
        {'hostname': 'xyz', 'ip': '127.0.0.5', 'extra': 'check-it'},
        {'hostname': 'ijk', 'ip': '127.0.0.3', 'extra': 'check-this'},
        {'hostname': 'uvw', 'ip': '127.0.0.4', 'extra': 'check-this'},
        {'hostname': 'xyz', 'ip': '127.0.0.5', 'extra': 'check-this'}
    ]
}

CodePudding user response：

After extracting all possible values, you can combine them using itertools.product:

hostnames = []
ips = []
extras = []

for host in d['hosts']:
    hostnames.extend(host['hostname'].split(','))
    ips.extend(host['ip'].split(','))
    extras.extend(host['extra'].split(','))

result = {'hosts': []}
for (hostname, ip, extra) in product(hostnames, ips, extras):
    result['hosts'].append({'hostname': hostname, 'ip': ip, 'extra': extra})

This produces a cartesian product which however means that you also get all combinations between hostnames and IPs. So you have to keep track of the hostname -> IP mapping:

hostnames = []
ips = {}
extras = []
for host in d['hosts']:
    extras.extend(host['extra'].split(','))
    new_hosts = host['hostname'].split(',')
    new_ips = host['ip'].split(',')
    hostnames.extend(new_hosts)
    host2ip = {hostname: ip for hostname, ip in zip(new_hosts, new_ips)}
    ips.update(host2ip)

result = {'hosts': []}
for (hostname, extra) in product(hostnames, extras):
    result['hosts'].append({'hostname': hostname, 'ip': ips[hostname], 'extra': extra})

This gives

{'hosts': [{'extra': 'check-me', 'hostname': 'ijk', 'ip': '127.0.0.3'},
           {'extra': 'check-this', 'hostname': 'ijk', 'ip': '127.0.0.3'},
           {'extra': 'check-it', 'hostname': 'ijk', 'ip': '127.0.0.3'},
           {'extra': 'check-me', 'hostname': 'uvw', 'ip': '127.0.0.4'},
           {'extra': 'check-this', 'hostname': 'uvw', 'ip': '127.0.0.4'},
           {'extra': 'check-it', 'hostname': 'uvw', 'ip': '127.0.0.4'},
           {'extra': 'check-me', 'hostname': 'xyz', 'ip': '127.0.0.5'},
           {'extra': 'check-this', 'hostname': 'xyz', 'ip': '127.0.0.5'},
           {'extra': 'check-it', 'hostname': 'xyz', 'ip': '127.0.0.5'}]}

CodePudding user response：

This produces the restricted product as in the sample output:

from itertools import product

dd = d['hosts'][0]
z = [*map(lambda x: x.split(','), dd.values())]
{'hosts': [dict(zip(dd.keys(), (*x, y))) for x, y in product(zip(*z[:2]), z[2])]}

Ouput:

{'hosts': [{'hostname': 'ijk', 'ip': '127.0.0.3', 'extra': 'check-me'},
           {'hostname': 'ijk', 'ip': '127.0.0.3', 'extra': 'check-this'},
           {'hostname': 'ijk', 'ip': '127.0.0.3', 'extra': 'check-it'},
           {'hostname': 'uvw', 'ip': '127.0.0.4', 'extra': 'check-me'},
           {'hostname': 'uvw', 'ip': '127.0.0.4', 'extra': 'check-this'},
           {'hostname': 'uvw', 'ip': '127.0.0.4', 'extra': 'check-it'},
           {'hostname': 'xyz', 'ip': '127.0.0.5', 'extra': 'check-me'},
           {'hostname': 'xyz', 'ip': '127.0.0.5', 'extra': 'check-this'},
           {'hostname': 'xyz', 'ip': '127.0.0.5', 'extra': 'check-it'}]}

On the other hand, this gives the true product for all keys:

combos = product(*(s.split(',') for s in d['hosts'][0].values()))
{'hosts': [dict(zip(d['hosts'][0].keys(), c)) for c in combos]}