I am trying to use the default filter in Jinja2 to replace missing values with default values.

It turns out that my data source has some NaNs, not just missing values. For my use case, I want to replace NaNs with a default value. But the default filter just passes NaN through without substitution.

Are there any filters in Jinja2 which would allow me to replace NaN values with something else, easily? (Of course I could write a custom filter, I'm just wondering if there's a simpler solution.)

For context, I'm doing this from Ansible, which has a few more filters available than plain Jinja2.

MWE

playbook.yaml

---
- hosts: localhost
  connection: local
  tasks:
  - name: generate cloudformation template
    template:
      src: input.j2
      dest: output.txt
    vars:
      data:
        - value: 123
          comment: this is an integer
        - value: None
          comment: This is None
        - value: NaN
          comment: This is NaN
        - comment: this is missing
        - value: ''
          comment: empty string

input.j2

{% for row in data %}
{{ row['comment'] }}
{{ row['value'] | default('default') }}
{{ row['value'] | default('default', boolean=True) }}

{% endfor %}

Run with:

ansible-playbook playbook.yaml
cat output.txt

Actual behavior

this is an integer
123
123
123

This is None
None
None
default

This is NaN
NaN
default
default

this is missing
default
default
default

empty string

default
default

Desired behavior

...

This is NaN
NaN
default
default

...

(It would be good to also have a solution that replaces Nones. I'm not sure why None | default('x', boolean=True) doesn't do that, since None is falsey.)

CodePudding user response：

The default filter won't help you here. A custom filter might be a good idea to simplify the code. But, you can convert the data also using current filters.

For example, add the default value first if the attribute is missing

_data_default:
  - value: default
_data: "{{ _data_default|product(data)|map('combine')|list }}"

_data:
  - comment: this is an integer
    value: 123
  - comment: This is None
    value: None
  - comment: This is NaN
    value: NaN
  - comment: this is missing
    value: default
  - comment: empty string
    value: ''

Replace the values

_regex: '^NaN|None|null$'
_regex_empty: '^$'
_replace: default
_data_values: "{{ _data|map(attribute='value')|
                  map('regex_replace', _regex, _replace)|
                  map('regex_replace', _regex_empty, _replace)|
                  map('community.general.dict_kv', 'value')|list }}"

_data_values:
  - value: '123'
  - value: default
  - value: default
  - value: default
  - value: default

and combine the items of the lists

data2: "{{ data|zip(_data_values)|map('combine')|list }}"

data2:
  - comment: this is an integer
    value: '123'
  - comment: This is None
    value: default
  - comment: This is NaN
    value: default
  - comment: this is missing
    value: default
  - comment: empty string
    value: default

Fit the options to your needs and put the variables as appropriate.