I want to loop through and combine fields of a lookup/csv file into a list.

Below is a sample CSV file and read into a list (hosts_list)

id,hostname,host_ip,country_code,country_name
ID01,myhost1,10.2.3.2,US,United States
ID02,myhost2,10.2.3.3,US,United States
ID03,myhost3,10.2.3.4,UK,United Kingdom
ID04,myhost3,10.2.3.4,US,United Kingdom

Expected output is

['US-myhost1', 'US-myhost2', 'US-myhost3']

I've done it in two methods

### Method1 - Looping the list. Works good on small subset, but highly inefficient for large fields

- name: "Read as a list"
  set_fact:
    my_clubbed_list: "{{ my_clubbed_list|default([])  [ my_clubbed_field ]}}"
  loop: "{{ hosts_list.list }}"
  vars:
    - my_clubbed_field: "{{item.country_code}}-{{item.hostname}}"
  when: item.country_code == 'US'

### Method2 - Using Jinja loop. Works fast but bit ugly

    - debug:
        msg: '{{ clubbed }}'
      vars:
        clubbed: |
          " 
            {%- for result in hosts_list.list -%}
            {% if 'US' == result.country_code %}
            '{{ result.country_code }}-{{ result.hostname }}',
            {% endif %}
            {% endfor %}
          "

Is there a better/efficient way to do this?

CodePudding user response：

In a nutshell:

---
- hosts: localhost
  gather_facts: false

  vars:
    hosts_query: >-
      [][country_code,hostname].join('-', @)
    hosts_list: "{{ hosts_csv.list | d([]) | json_query(hosts_query) }}"

  tasks:
    - name: Get CSV content
      ansible.builtin.read_csv:
        path: files/hosts.csv
      register: hosts_csv

    - name: Show calculated hosts list
      ansible.builtin.debug:
        var: hosts_list

If you don't have jmespath/json_query available, you can eventually replace the vars section above with:

  vars:
    hosts_codes: "{{ hosts_csv.list | d([]) | map(attribute='country_code') }}"
    hosts_names: "{{ hosts_csv.list | d([]) | map(attribute='hostname') }}"
    hosts_list: "{{ hosts_codes | zip(hosts_names) | map('join', '-') }}"

With both scenario and using your above CSV content we get:

TASK [Get CSV content] *****************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Show calculated hosts list] ******************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "hosts_list": [
        "US-myhost1",
        "US-myhost2",
        "UK-myhost3",
        "US-myhost3"
    ]
}

PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

CodePudding user response：

Use the Jinja template. For example,

  clubbed_str: |
    {% for i in hosts_csv.list|groupby('country_code') %}
    {{ i.0 }}:
    {% for h in i.1|map(attribute='hostname') %}
      - {{ i.0 }}-{{ h }}
    {% endfor %}  
    {% endfor %}
  clubbed_dir: "{{ clubbed_str|from_yaml }}"

gives

  clubbed_dir:
    UK: [UK-myhost3]
    US: [US-myhost1, US-myhost2, US-myhost3]

- hosts: localhost

  vars:

    clubbed_str: |
      {% for i in hosts_csv.list|groupby('country_code') %}
      {{ i.0 }}:
      {% for h in i.1|map(attribute='hostname') %}
        - {{ i.0 }}-{{ h }}
      {% endfor %}  
      {% endfor %}
    clubbed_dir: "{{ clubbed_str|from_yaml }}"

  tasks:

    - name: Get CSV content
      ansible.builtin.read_csv:
        path: "{{ playbook_dir }}/files/hosts.csv"
      register: hosts_csv

    - debug:
        var: clubbed_dir|to_yaml
    - debug:
        var: clubbed_dir.US|to_yaml

PLAY [localhost] *****************************************************************************

TASK [Get CSV content] ***********************************************************************
ok: [localhost]

TASK [debug] *********************************************************************************
ok: [localhost] => 
  clubbed_dir|to_yaml: |-
    UK: [UK-myhost3]
    US: [US-myhost1, US-myhost2, US-myhost3]

TASK [debug] *********************************************************************************
ok: [localhost] => 
  clubbed_dir.US|to_yaml: |-
    [US-myhost1, US-myhost2, US-myhost3]

PLAY RECAP ***********************************************************************************
localhost: ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0