Using ansible playbook, and lookup plugins, How to convert the result of "curl https://ifconfig.io/all" which has the dictionary format key: value to item list

country_code: US
encoding: gzip
forwarded: 29.493.593.012
host: XXX.XXX.X53.2XX
ifconfig_hostname: ifconfig.io
ip: XXX.XXX.X53.2XX
lang: ""
method: GET
mime: '/'
port: 27404
referer: ""
ua: Python-urllib/3.6

The required output: items list in this format:

[
        {
            "key": "country_code",
            "value": "US"
        },
        {
            "key": "encoding",
            "value": "gzip"
        },
        {
            "key": "forwarded",
            "value": "29.493.593.012"
        },
        {
            "key": "host",
            "value": "XXX.XXX.X53.2XX"
        },
        {
            "key": "ifconfig_hostname",
            "value": "ifconfig.io"
        },
        {
            "key": "ip",
            "value": "XXX.XXX.X53.2XX"
        },
        {
            "key": "lang",
            "value": "''"
        },
        {
            "key": "method",
            "value": "GET"
        },
        {
            "key": "mime",
            "value": "''"
        },
        {
            "key": "port",
            "value": "28180"
        },
        {
            "key": "referer",
            "value": "''"
        },
        {
            "key": "ua",
            "value": "Python-urllib/3.6"
        }
    ]

The script I have used is:

- name: Get url contents             
  vars:                              
    curl_res: "{{ query('url','https://ifconfig.io/all') }}"
  debug:                             
    msg: "{{ item }}"                
  loop: "{{ curl_res | dict2items }}"     

But I have got this error:

TASK [Get url contents] *****************************************************************************************************************

fatal: [servera]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{ curl_res | dict2items }}): dict2items requires a dictionary, got <class 'list'> instead."}

So, I made it manually, saved the curl results in a local file then fine-tune it to this format:

cat files/curl_result

{
    "country_code": "US",
    "encoding": "gzip",
    "forwarded": "29.493.593.012",
    "host": "XXX.XXX.X53.2XX",
    "ifconfig_hostname": "ifconfig.io",
    "ip": "XXX.XXX.X53.2XX",
    "lang": "''",
    "method": "GET",
    "mime": "''",
    "port": "28580",
    "referer": "''",
    "ua": "Python-urllib/3.6"
}

then, I have run this task in my play :

- name: extract from file 
  vars: 
    items_curl_res: "{{ lookup('file', 'curl_result') | from_json | dict2items }}"
   debug:    
     var: items_curl_res

CodePudding user response：

The return you are getting is not a dictionary, but a list of string in the form key: value.

That form is actually a valid YAML format, so you could use map to apply a from_yaml filter to all those string, then, map again a dict2items and finally, flatten the list of list.

Ending up with this taks:

- debug:
   var: >-
      query('url','https://ifconfig.io/all') 
      | map('from_yaml') 
      | map('dict2items') 
      | flatten 

This task would yield:

query('url','https://ifconfig.io/all')  | map('from_yaml')  | map('dict2items')  | flatten:
- key: country_code
  value: BE
- key: encoding
  value: gzip
- key: forwarded
  value: 93.184.216.34
- key: host
  value: 93.184.216.34
- key: ifconfig_hostname
  value: ifconfig.io
- key: ip
  value: 93.184.216.34
- key: lang
  value: ''
- key: method
  value: GET
- key: mime
  value: ''
- key: port
  value: 24294
- key: referer
  value: ''
- key: ua
  value: ansible-httpget

CodePudding user response：

Wow .. it is working, thanks β.εηοιτ.βε:

here the working script:

- name: Get url contents                             
  debug:                                             
    msg: "{{ query('url','https://ifconfig.io/all') | map('from_yaml') | map('dict2items') | flatten }}"

and the output:

"msg": [
    {
        "key": "country_code",
        "value": "US"
    },
    {
        "key": "encoding",
        "value": "gzip"
    },
    {
        "key": "forwarded",
        "value": "XXX.XX3.8XX.X9X"
    },
    {
        "key": "host",
        "value": "XXX.XX3.8XX.X9X"
    },
    {
        "key": "ifconfig_hostname",
        "value": "ifconfig.io"
    },
    {
        "key": "ip",
        "value": "3XX.X7X.XX4.XX0"
    },
    {
        "key": "lang",
        "value": ""
    },
    {
        "key": "method",
        "value": "GET"
    },
    {
        "key": "mime",
        "value": ""
    },
    {
        "key": "port",
        "value": 33470
    },
    {
        "key": "referer",
        "value": ""
    },
    {
        "key": "ua",
        "value": "Python-urllib/3.6"
    }
]