Home > Enterprise >  JOLT transformation build array of objects grouping attributes in the same object
JOLT transformation build array of objects grouping attributes in the same object

Time:06-01

I'm trying to construct an array of objects named catalogs with different combinations of input values, but can't set some attributes in the same object. This is the current progress:

[
  {
    "operation": "shift",
    "spec": {
      "new": {
        "bc_sku_channel": {
          "*": {
            "Partner": null, //ignore if value is Partner 
            "*": {
              "#BC": "catalogs[#3].catalog",
              "@1": "catalogs[#3].channel"
            }
          }
        },
        "bc_sku_partner": {
          "*": {
            "#BC": "catalogs[].catalog", // don't know which index should be here to group these 3 attributes into the same object
            "#Partner": "catalogs[].channel", // tried #3,#2,#1,#4 but doesn't work
            "@": "catalogs[].partner"
          }
        },
        "cc_sku_channel": {
          "*": {
            "Partner": null, //ignore if value is Partner 
            "*": {
              "#CC": "catalogs[#3].catalog",
              "@1": "catalogs[#3].channel"
            }
          }
        }
      }
    }
  }
]

Input:

{
  "new": {
    "bc_sku_partner": [
      "Amazon",
      "Ebay"
    ],
    "bc_sku_channel": [
      "Partner",
      "Online",
      "Store"
    ],
    "cc_sku_channel": [
      "Store"
    ]
  }
}

Expected Output:

 {
  "catalogs": [
    {
      "catalog": "BC",
      "channel": "Partner",
      "partner": "Amazon"
    },
    {
      "catalog": "BC",
      "channel": "Partner",
      "partner": "Ebay"
    },
    {
      "catalog": "BC",
      "channel": "Online"
    },
    {
      "catalog": "BC",
      "channel": "Store"
    },
    {
      "catalog": "CC",
      "channel": "Store"
    }
  ]
}

Now I have only been able to build the last 3 objects.

Notes:

If channel is Online or Store the object shouldn't have the partner attribute.

For each partner, the channel attribute should be always Partner

CodePudding user response:

This spec produces the expected output

Version which assumes that bc_ and cc_ prefixes are dynamic:

Simpler version can be found at the end of this answer.

[
  {
    "operation": "shift",
    "spec": {
      "new": {
        "*_sku_partner": {      // 1
          "*": {                // 2
            "*": {              // 3
              "@1": {           // 4
                "$1": "partner__&1__&(4,1).partner",              // 5
                "#Partner": "partner__&1__&(4,1).channel",        // 6
                "$(3,1)": "partner__&1__&(4,1).catalogLowerCase"  // 7
              }
            }
          }
        },
        "*_sku_channel": {
          "*": {
            "Partner": null,        // 8
            "*": {
              "@1": {
                "$1": "channel__&1__&(4,1).channel",
                "$(3,1)": "channel__&1__&(4,1).catalogLowerCase"
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "shift",            // 9
    "spec": {
      "partner__*": "catalogs[#1]",
      "channel__*": "catalogs[#1]"
    }
  },
  {
    "operation": "modify-default-beta",  // 10
    "spec": {
      "catalogs": {
        "*": {
          "catalog": "=toUpper(@(1,catalogLowerCase))"
        }
      }
    }
  },
  {
    "operation": "shift",  // 11
    "spec": {
      "catalogs": {
        "*": {
          "catalogLowerCase": null,
          "*": "&2[&1].&"
        }
      }
    }
  }
]

The first operation is the trickiest, I guess: It creates a dictionary with unique entries, like:

{
  "channel__Online__bc" : {
    "catalogLowerCase" : "bc",
    "channel" : "Online"
  }
  // ...
  "partner__Amazon__bc": {
    "catalogLowerCase": "bc",
    "channer": "Partner",
    "partner": "Amazon"
  }
}

from:

// ...
"bc_sku_partner": [
  "Amazon",
  "Ebay"
]
// ...

Let's focus on the first entry generated for "Amazon":

Ad. 1. *_sku_partner matches "bc_sku_partner". You could also replace the asterisk with the explicit bc value and instead of $(3,1) in ad. 7 place #BC

Ad. 2. Matches the elements of the ["Amazon", ...] list.

Ad. 3. We are now inside "Amazon" :D

Ad. 4. We go into "Amazon" and reach for it (@1). Related docs: @ shift wildcard

Ad. 5. Looking at the ($ wildcard docs)

'$' has the same syntax as the '&' wildcard, and can be read as, dereference to get a value, and then use that value as the data to be output.

$1 goes up one level, grabs the value ("Amazon") and places it under they "partner__&1__&(4,1).partner: key. "partner__&1__&(4,1)" for the "Amazon" element generates: "partner__Amazon__bc". &(4,1) means here:

Go up 4 levels (we are at "bc_sku_partner") and take the value matched by the asterisk. See: & wildcard docs

Ad. 6. Places "Partner" string under the "channel" key (next to the "partner": "Amazon" key-value).

Ad. 7. $(3,1) again but here we reach for the "bc" matched by the asterisk and place it under the "catalogLowerCase" key.

The "*_sku_channel" part is quite similar, so I am not going to explain it. At ad. 8. is the removal of the "Partner" entries.

Ad. 9. The second shift operation. It's purpose is to transform the dictionary produced by the 1st shift operation into the list placed under the "catalogs" key. # wildcard docs may be useful here.

Ad. 10 and 11 replace the "catalogLowerCase": "bc" with "catalog": "BC". Same for the similar "catalogLowerCase" fields related to the "Ebay", "Online" and "Store" entries.

Try to apply the operations incrementally, to see the intermediate result. I recommend this website for testing: https://jolt-demo.appspot.com/. It also has a lot of examples.


Edit:

Other way to do it, if the bc_ and cc_ prefixes are not dynamic.

[
  {
    "operation": "shift",
    "spec": {
      "new": {
        "bc_sku_partner": { // 1
          "*": { // 2
            "*": { // 3
              "@1": { // 4
                "$1": "partner__&1__bc.partner", // 5
                "#Partner": "partner__&1__bc.channel", // 6
                "#BC": "partner__&1__bc.catalog" // 7
              }
            }
          }
        },
        "bc_sku_channel": {
          "*": {
            "Partner": null,
            "*": {
              "@1": {
                "$1": "channel__&1__bc.channel",
                "#BC": "channel__&1__bc.catalog"
              }
            }
          }
        },
        "cc_sku_channel": {
          "*": {
            "Partner": null,
            "*": {
              "@1": {
                "$1": "channel__&1__cc.channel",
                "#CC": "channel__&1__cc.catalog"
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "partner__*": "catalogs[#1]",
      "channel__*": "catalogs[#1]"
    }
  }
]
  • Related