How to iterate through the data of a column in a sql case statement in dbt?-CodePudding

Newbie in dbt here.

I need to do a case statement, something like this:


case when PROPERTY_NAME = 'xxx' and EVENT_CATEGORY = 'xxx' and EVENT_ACTION LIKE '%xxx%' and EVENT_LABEL like '%xxx%'
                then 'xxx' 
(...)

on property name, I need to iterate through a list of data from a column of a table.

Is it doable though a macro?

CodePudding user response：

I think the best strategy depends on what you want to do. If all you need is the list of all columns, you could use something like get_columns_in_relation and use the results to loop over in your case statement:

{%- set columns = adapter.get_columns_in_relation(ref('model')) -%}

case
  when
  {% for column in columns %}
  {{column.name}} = 'xxx' {{ 'and' if not loop.last }}
  {% endfor %}
...

If you don't need every column, you could either exclude some columns from the resulting list or (better IMO) just define the columns you need in a jinja variable and loop over those.

If you need the data from one of the columns, you can use the (similar) run_query macro (or the get_column_values macro in dbt-utils). These have the same pattern of use - ie. retrieve something into the jinja layer of dbt and then use that layer to template out some sql.

CodePudding user response：

To get data into the jinja context, you can use the run_query macro to execute arbitrary SQL and return the results as an Agate table. There is an example in the dbt docs for how to use that to return the distinct values from a column.

This use case is so common, there is also a macro in dbt-utils for it called get_column_values. This macro returns the distinct values from a column as an array (list). The example from those docs:

-- Returns a list of the payment_methods in the stg_payments model_
{% set payment_methods = dbt_utils.get_column_values(table=ref('stg_payments'), column='payment_method') %}

{% for payment_method in payment_methods %}
    ...
{% endfor %}

...

(Note that you need to first install the dbt-utils package. Docs for that here).

If you are trying to check membership in this list of values, you could do something like this:

{% set properties = dbt_utils.get_column_values(table=ref('your_model'), column='property') %}
{% set properties_str = properties | join("', '") %}
case
    when 
        PROPERTY_NAME in ('{{ properties_str }}')
        and EVENT_CATEGORY = 'xxx' 
        and EVENT_ACTION LIKE '%xxx%' 
        and EVENT_LABEL like '%xxx%'
    then 'xxx'
...

Or if you want to iterate over that list:

{% set properties = dbt_utils.get_column_values(table=ref('your_model'), column='property') %}
case
    {% for property in properties %}
    when 
        PROPERTY_NAME = '{{ property }}'
        and EVENT_CATEGORY = 'xxx' 
        and EVENT_ACTION LIKE '%xxx%' 
        and EVENT_LABEL like '%xxx%'
    then '{{ property }}'
    {% endfor %}
...