Home > Net >  Ansible: Combining 2 Lists by a Specific Attribute
Ansible: Combining 2 Lists by a Specific Attribute

Time：07-14

I have 2 lists that I'm trying to combine together, by a specific attribute (the "device" attribute). I am not instantiating the lists in my code anywhere, they are being populated by API calls so the best I have here are printouts of each list.

List1:

TASK [Print List1] **************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool"
            },
            {
                "device": "LTM2_Device",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool"
            },
            {
                "device": "LTM3_Device",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool"
            }
        ]
    ]
}

List2:

TASK [Print List2] ****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host1",
                "ip": "0.0.0.1",
                "port": "5555"
            },
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host2",
                "ip": "0.0.0.2",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host1",
                "ip": "0.0.0.3",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.4",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host1",
                "ip": "0.0.0.5",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host2",
                "ip": "0.0.0.6",
                "port": "5555"
            }
        ]
    ]
}

I have 2 debug lines that print something fairly close to what I am looking for, however neither are quite what I want... and I can't seem to get the Jinja2 filter correct.

- name: Debug
    debug:
        msg: 
            - "{{ (list2   list1) | groupby('device') | map('last') | map('combine') | list }}"
            - "{{ (list2   list1) | groupby('device') | map('last') | list }}"

Which gives the following output:

TASK [Debug] ****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host2",
                "ip": "0.0.0.1",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.3",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.5",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool",
                "port": "5555"
            }
        ],
        [
            [
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host1",
                    "ip": "0.0.0.1",
                    "port": "5555"
                },
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host2",
                    "ip": "0.0.0.2",
                    "port": "5555"
                },
                {
                    "device": "LTM1_Device",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool"
                }
            ],
            [
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host1",
                    "ip": "0.0.0.3",
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host2",
                    "ip": "0.0.0.4",
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool"
                }
            ],
            [
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host1",
                    "ip": "0.0.0.5",
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host2",
                    "ip": "0.0.0.6",
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool"
                }
            ]
        ]
    ]
}

The first debug line does not combine both lists for all elements of list2 (it misses the first host for each LTM), and the second line combines them as a separate element in a nested list. Is there a way to combine both lists so it would look something like the following? I know I'm probably missing something small but I can't seem to figure out what it is I'm missing. Any help would be greatly appreciated, thanks!

Desired output (I'm not worried about it being a nested list as I can flatten that later):

        [
            [
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host1",
                    "ip": "0.0.0.1",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool"
                    "port": "5555"
                },
                {
                    "device": "LTM1_Device",
                    "host": "/Common/LTM1_Host2",
                    "ip": "0.0.0.2",
                    "link": "LTM1_Link",
                    "ltm_pool": "LTM1_Pool"
                    "port": "5555"
                }
            ],
            [
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host1",
                    "ip": "0.0.0.3",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool"
                    "port": "5555"
                },
                {
                    "device": "LTM2_Device",
                    "host": "/Common/LTM2_Host2",
                    "ip": "0.0.0.4",
                    "link": "LTM2_Link",
                    "ltm_pool": "LTM2_Pool"
                    "port": "5555"
                }
            ],
            [
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host1",
                    "ip": "0.0.0.5",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool"
                    "port": "5555"
                },
                {
                    "device": "LTM3_Device",
                    "host": "/Common/LTM3_Host2",
                    "ip": "0.0.0.6",
                    "link": "LTM3_Link",
                    "ltm_pool": "LTM3_Pool"
                    "port": "5555"
                }
            ]
        ]

CodePudding user response：

There are more options.

  • Iterate the lists if list1 is properly sorted and provides exactly what you need
    - set_fact:
        result: "{{ result|d([])   _item }}"
      with_together:
        - "{{ list1.0 }}"
        - "{{ list2.0|groupby('device')|map(attribute=1)|list }}"
      vars:
        _item: "{{ [item.0]|product(item[1:])|map('combine')|list }}"

gives the desired result

result:
  - device: LTM1_Device
    host: /Common/LTM1_Host1
    ip: 0.0.0.1
    link: LTM1_Link
    ltm_pool: LTM1_Pool
    port: '5555'
  - device: LTM1_Device
    host: /Common/LTM1_Host2
    ip: 0.0.0.2
    link: LTM1_Link
    ltm_pool: LTM1_Pool
    port: '5555'
  - device: LTM2_Device
    host: /Common/LTM2_Host1
    ip: 0.0.0.3
    link: LTM2_Link
    ltm_pool: LTM2_Pool
    port: '5555'
  - device: LTM2_Device
    host: /Common/LTM2_Host2
    ip: 0.0.0.4
    link: LTM2_Link
    ltm_pool: LTM2_Pool
    port: '5555'
  - device: LTM3_Device
    host: /Common/LTM3_Host1
    ip: 0.0.0.5
    link: LTM3_Link
    ltm_pool: LTM3_Pool
    port: '5555'
  - device: LTM3_Device
    host: /Common/LTM3_Host2
    ip: 0.0.0.6
    link: LTM3_Link
    ltm_pool: LTM3_Pool
    port: '5555'

(details)

  • The next option is converting list1 to dictionaries
links: "{{ list1.0|items2dict(key_name='device', value_name='link') }}"
pools: "{{ list1.0|items2dict(key_name='device', value_name='ltm_pool') }}"

give

links:
  LTM1_Device: LTM1_Link
  LTM2_Device: LTM2_Link
  LTM3_Device: LTM3_Link
pools:
  LTM1_Device: LTM1_Pool
  LTM2_Device: LTM2_Pool
  LTM3_Device: LTM3_Pool

Iterate list2 and combine the dictionaries. The task below gives also the desired result

    - set_fact:
        result: "{{ result|d([])   [_item] }}"
      loop: "{{ list2.0 }}"
      vars:
        _item: "{{ item|combine({'link': links[item.device]})|
                        combine({'ltm_pool': pools[item.device]}) }}"

(details)

Notes

  1. Example of a complete playbook
- hosts: localhost
  vars:
    list1:
      - - device: LTM1_Device
          link: LTM1_Link
          ltm_pool: LTM1_Pool
        - device: LTM2_Device
          link: LTM2_Link
          ltm_pool: LTM2_Pool
        - device: LTM3_Device
          link: LTM3_Link
          ltm_pool: LTM3_Pool
    list2:
      - - device: LTM1_Device
          host: /Common/LTM1_Host1
          ip: 0.0.0.1
          port: '5555'
        - device: LTM1_Device
          host: /Common/LTM1_Host2
          ip: 0.0.0.2
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host1
          ip: 0.0.0.3
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host2
          ip: 0.0.0.4
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host1
          ip: 0.0.0.5
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host2
          ip: 0.0.0.6
          port: '5555'
    links: "{{ list1.0|items2dict(key_name='device', value_name='link') }}"
    pools: "{{ list1.0|items2dict(key_name='device', value_name='ltm_pool') }}"
  tasks:
    - set_fact:
        result: "{{ result|d([])   [_item] }}"
      loop: "{{ list2.0 }}"
      vars:
        _item: "{{ item|combine({'link': links[item.device]})|
                        combine({'ltm_pool': pools[item.device]}) }}"
    - debug:
        var: result
  1. If you want to avoid iteration create lists of the attributes ip and device
ip: "{{ list2.0|map(attribute='ip')|list }}"
device: "{{ list2.0|map(attribute='device')|list }}"

give

ip: [0.0.0.1, 0.0.0.2, 0.0.0.3, 0.0.0.4, 0.0.0.5, 0.0.0.6]
device: [LTM1_Device, LTM1_Device, LTM2_Device, LTM2_Device, LTM3_Device, LTM3_Device]

Then use the list device and extract both links and pools

link: "{{ device|map('extract', links)|list }}"
pool: "{{ device|map('extract', pools)|list }}"

give

link: [LTM1_Link, LTM1_Link, LTM2_Link, LTM2_Link, LTM3_Link, LTM3_Link]
pool: [LTM1_Pool, LTM1_Pool, LTM2_Pool, LTM2_Pool, LTM3_Pool, LTM3_Pool]

Create dictionaries

link_dict: "{{ dict(ip|zip(link)) }}"
pool_dict: "{{ dict(ip|zip(pool)) }}"

give

link_dict:
  0.0.0.1: LTM1_Link
  0.0.0.2: LTM1_Link
  0.0.0.3: LTM2_Link
  0.0.0.4: LTM2_Link
  0.0.0.5: LTM3_Link
  0.0.0.6: LTM3_Link
pool_dict:
  0.0.0.1: LTM1_Pool
  0.0.0.2: LTM1_Pool
  0.0.0.3: LTM2_Pool
  0.0.0.4: LTM2_Pool
  0.0.0.5: LTM3_Pool
  0.0.0.6: LTM3_Pool

and convert the dictionaries to lists

link_list: "{{ link_dict|dict2items(key_name='ip', value_name='link') }}"
pool_list: "{{ pool_dict|dict2items(key_name='ip', value_name='ltm_pool') }}"

give

link_list:
  - {ip: 0.0.0.1, link: LTM1_Link}
  - {ip: 0.0.0.2, link: LTM1_Link}
  - {ip: 0.0.0.3, link: LTM2_Link}
  - {ip: 0.0.0.4, link: LTM2_Link}
  - {ip: 0.0.0.5, link: LTM3_Link}
  - {ip: 0.0.0.6, link: LTM3_Link}
pool_list:
  - {ip: 0.0.0.1, ltm_pool: LTM1_Pool}
  - {ip: 0.0.0.2, ltm_pool: LTM1_Pool}
  - {ip: 0.0.0.3, ltm_pool: LTM2_Pool}
  - {ip: 0.0.0.4, ltm_pool: LTM2_Pool}
  - {ip: 0.0.0.5, ltm_pool: LTM3_Pool}
  - {ip: 0.0.0.6, ltm_pool: LTM3_Pool}

Finally, use filter community.general.lists_mergeby and merge the lists by attribute ip. This gives the desired result

result: "{{ [list2.0, link_list, pool_list]|community.general.lists_mergeby('ip') }}"

Example of a complete playbook

- hosts: localhost
  vars:
    list1:
      - - device: LTM1_Device
          link: LTM1_Link
          ltm_pool: LTM1_Pool
        - device: LTM2_Device
          link: LTM2_Link
          ltm_pool: LTM2_Pool
        - device: LTM3_Device
          link: LTM3_Link
          ltm_pool: LTM3_Pool
    list2:
      - - device: LTM1_Device
          host: /Common/LTM1_Host1
          ip: 0.0.0.1
          port: '5555'
        - device: LTM1_Device
          host: /Common/LTM1_Host2
          ip: 0.0.0.2
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host1
          ip: 0.0.0.3
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host2
          ip: 0.0.0.4
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host1
          ip: 0.0.0.5
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host2
          ip: 0.0.0.6
          port: '5555'
    links: "{{ list1.0|items2dict(key_name='device', value_name='link') }}"
    pools: "{{ list1.0|items2dict(key_name='device', value_name='ltm_pool') }}"
    ip: "{{ list2.0|map(attribute='ip')|list }}"
    device: "{{ list2.0|map(attribute='device')|list }}"
    link: "{{ device|map('extract', links)|list }}"
    pool: "{{ device|map('extract', pools)|list }}"
    link_dict: "{{ dict(ip|zip(link)) }}"
    pool_dict: "{{ dict(ip|zip(pool)) }}"
    link_list: "{{ link_dict|dict2items(key_name='ip', value_name='link') }}"
    pool_list: "{{ pool_dict|dict2items(key_name='ip', value_name='ltm_pool') }}"
    result: "{{ [list2.0, link_list, pool_list]|community.general.lists_mergeby('ip') }}"
  tasks:
    - debug:
        var: result

(details)

  1. If you can't or don't want to use community.general.lists_mergeby zip the lists and combine items. This gives also the desired result
result: "{{ list2.0|zip(link_list)|zip(pool_list)|map('flatten')|map('combine')|list }}"

CodePudding user response：

With the help of a peer, I was able to figure it out by looping the second list, adding a selectattr filter to match my attribute against the first list, and combining that result with the second list to create a new list of results.

Filtered Loop:

- name: Combine lists
  set_fact:
    new_list: "{{ new_list | default([])   [item|combine(filter_time)] }}"
  loop: "{{ list2 }}"
  vars:
    filter_time: "{{ list1 | selectattr('device', '==', item.device) | first }}"

Output:

ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host1",
                "ip": "0.0.0.1",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool",
                "port": "5555"
            },
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host2",
                "ip": "0.0.0.2",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host1",
                "ip": "0.0.0.3",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.4",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host1",
                "ip": "0.0.0.5",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host2",
                "ip": "0.0.0.6",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool",
                "port": "5555"
            }
        ]
    ]
}

Page link：https//www.codepudding.com/net/475039.html

Prev:Search elements in a list in another list and check each element exist or not
Next:Trying to return filetype for all users appdata using PowerShell
  • Related

About Us:  Contact Us      Terms of Service       Privacy Policy

Copyright © 2010-2023，Powered By CodePudding