Home > Mobile >  Extract and manipulate dict data to check certificates
Extract and manipulate dict data to check certificates

Time:12-19

I struggle on a regular basis with data manipulation in Ansible. I'm not very familiar with Python and dict objects. I found an example that sums up a lot of my misunderstandings.

I would like to verify a list of certificates. In found an example for a single domain in the documentation, I'm just trying to loop over several domain names.

Certs are stored in a folder:

certs/
├── domain.com
│   ├── domain.com.pem
│   └── domain.com.key
└── domain.org
    ├── domain.org.key
    └── domain.org.pem

My playbook is as follow:

---
- name: "check certs"
  hosts: localhost
  gather_facts: no
  vars:
    domain_names:
      - domain.com
      - domain.org
    certs_folder: certs
  tasks: 
    - name: Get certificate information
      community.crypto.x509_certificate_info:
        path: "{{ certs_folder }}/{{ item }}/{{ item }}.pem"
        # for valid_at, invalid_at and valid_in
      register: result_certs
      loop: "{{ domain_names }}"
      failed_when: 0

    - name: Get private key information
      community.crypto.openssl_privatekey_info:
        path: "{{ certs_folder }}/{{ item }}/{{ item }}.key"
      register: result_privatekey
      loop: "{{ domain_names }}"
      failed_when: 0

    - name: Check cert and key match << DOES NOT WORK >>>
      assert: 
        that:
          - result_certs[ {{ item }} ].public_key == result_privatekey[ {{ item }} ].public_key
          # - ... other checks ...
          - not result[ {{ item }} ].expired
      loop: "{{ domain_names }}"

So I get two variables result_certs and result_privatekey, each has a element result which is , if I understand correctly, an array of dicts:

"result_certs": {
        "changed": false,
        "msg": "All items completed",
        "results": [
            {
                "expired": false,
                "item": "domain.org",
                "public_key": "<<PUBLIC KEY>>"
            },
            {
                "expired": false,
                "item": "domain.com",
                "public_key": "<<PUBLIC KEY>>"
            }
        ],
        "skipped": false
    }
"result_privatekey": {
    "changed": false,
    "msg": "All items completed",
    "results": [
    {
        "item": "domain.org",
        "public_key": "<< PUBLIC KEY >>"
    },
    {
        "item": "domain.com",
        "public_key": "<< PUBLIC KEY >>"
    }
    ],
    "skipped": false
}

How can I refer to each of the dicts elements like result_privatekey.results[the dict where item ='domain.org'].public_key in the assert task?

I feel like I'm missing something, or a documentation page to make it clear to me. I noticed that I particularly struggle with arrays of dicts, and I run into those objects quite often...

I found those resources useful, but not sufficient to get this job done:

CodePudding user response:

From what you are showing — but you might have more conditions that needs it — I wouldn't loop on the domain_names in your assertion task, I would rather loop on result_certs.

From there on, you can select the corresponding private key thanks to the selectattr filter.

So, your assertion would become:

- assert: 
    that:
      - >-
          item.public_key == (
            result_privatekey.results 
              | selectattr('item', '==', item.item) 
              | first
          ).public_key
      # - ... other checks ...
      - not item.expired
  loop: "{{ result_certs.results }}"

CodePudding user response:

Given the simplified data for testing

  result_certs:
    changed: false
    msg: All items completed
    results:
      - expired: false
        item: domain.org
        public_key: <<PUBLIC KEY domain.org>>
      - expired: false
        item: domain.com
        public_key: <<PUBLIC KEY domain.com>>
    skipped: false

  result_privatekey:
    changed: false
    msg: All items completed
    results:
      - item: domain.org
        public_key: <<PUBLIC KEY domain.org>>
      - item: domain.com
        public_key: <<PUBLIC KEY domain.com>>
    skipped: false

Declare the list of the domains

  domains: "{{ result_certs.results|
               map(attribute='item')|list }}"

gives

  domains:
  - domain.org
  - domain.com

Q: "How can I refer to each dictionary element?"

A: select the item(s) and map the attribute

    - debug:
        var: pk
      loop: "{{ domains }}"
      vars:
        pk: "{{ result_privatekey.results|
                selectattr('item', '==', item)|
                map(attribute='public_key')|list }}"

gives

TASK [debug] *********************************************************************************
ok: [localhost] => (item=domain.org) => 
  ansible_loop_var: item
  item: domain.org
  pk:
  - <<PUBLIC KEY domain.org>>
ok: [localhost] => (item=domain.com) => 
  ansible_loop_var: item
  item: domain.com
  pk:
  - <<PUBLIC KEY domain.com>>

It's practical to create a unique list of all keys

  pkeys: "{{ result_certs.results|
             zip(result_privatekey.results)|
             map('map', attribute='public_key')|
             map('unique')|flatten }}"

gives

  pkeys:
  - <<PUBLIC KEY domain.org>>
  - <<PUBLIC KEY domain.com>>

Compare the lengths of the lists if you want to know if there are redundant keys

  pkeys|length == domains|length

To find expired domains declare variables

    expired: "{{ result_certs.results|
                 map(attribute='expired')|list }}"
    expired_domains: "{{ result_certs.results|
                         selectattr('expired')|
                         map(attribute='item')|list }}"

give

  expired:
  - false
  - false

  expired_domains: []

Then the assert task should look like

  - assert:
      that:
        - expired is not any
        - pkeys|length == domains|length

Example of a complete playbook for testing

- hosts: localhost

  vars:

    result_certs:
      changed: false
      msg: All items completed
      results:
      - expired: false
        item: domain.org
        public_key: <<PUBLIC KEY domain.org>>
      - expired: false
        item: domain.com
        public_key: <<PUBLIC KEY domain.com>>
      skipped: false

    result_privatekey:
      changed: false
      msg: All items completed
      results:
      - item: domain.org
        public_key: <<PUBLIC KEY domain.org>>
      - item: domain.com
        public_key: <<PUBLIC KEY domain.com>>
      skipped: false

    domains: "{{ result_certs.results|
                 map(attribute='item')|list }}"
    pkeys: "{{ result_certs.results|
               zip(result_privatekey.results)|
               map('map', attribute='public_key')|
               map('unique')|flatten }}"
    expired: "{{ result_certs.results|
                 map(attribute='expired')|list }}"
    expired_domains: "{{ result_certs.results|
                         selectattr('expired')|
                         map(attribute='item')|list }}"

  tasks:

    - debug:
        var: domains

    - debug:
        var: pkeys

    # How can I refer to each of the dicts elements?
    - debug:
        var: pk
      loop: "{{ domains }}"
      vars:
        pk: "{{ result_privatekey.results|
                selectattr('item', '==', item)|
                map(attribute='public_key')|list }}"

    # How can I compare private keys?
    - debug:
        msg: "{{ pk1 }} == {{ pk2 }}: {{ pk1 == pk2 }}"
      loop: "{{ domains }}"
      vars:
        pk1: "{{ result_privatekey.results|
                 selectattr('item', '==', item)|
                 map(attribute='public_key')|first }}"
        pk2: "{{ result_certs.results|
                 selectattr('item', '==', item)|
                 map(attribute='public_key')|first }}"
    - debug:
        var: expired
    - debug:
        var: expired_domains

    - assert:
        that:
          - expired is not any
          - pkeys|length == domains|length

gives

PLAY [localhost] *****************************************************************************

TASK [debug] *********************************************************************************
ok: [localhost] => 
  domains:
  - domain.org
  - domain.com

TASK [debug] *********************************************************************************
ok: [localhost] => 
  pkeys:
  - <<PUBLIC KEY domain.org>>
  - <<PUBLIC KEY domain.com>>

TASK [debug] *********************************************************************************
ok: [localhost] => (item=domain.org) => 
  ansible_loop_var: item
  item: domain.org
  pk:
  - <<PUBLIC KEY domain.org>>
ok: [localhost] => (item=domain.com) => 
  ansible_loop_var: item
  item: domain.com
  pk:
  - <<PUBLIC KEY domain.com>>

TASK [debug] *********************************************************************************
ok: [localhost] => (item=domain.org) => 
  msg: '<<PUBLIC KEY domain.org>> == <<PUBLIC KEY domain.org>>: True'
ok: [localhost] => (item=domain.com) => 
  msg: '<<PUBLIC KEY domain.com>> == <<PUBLIC KEY domain.com>>: True'

TASK [debug] *********************************************************************************
ok: [localhost] => 
  expired:
  - false
  - false

TASK [debug] *********************************************************************************
ok: [localhost] => 
  expired_domains: []

TASK [assert] ********************************************************************************
ok: [localhost] => changed=false 
  msg: All assertions passed

PLAY RECAP ***********************************************************************************
localhost: ok=7    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
  • Related