JOLT shift transformation: collect all the items from all the levels without knowing how many levels-CodePudding

I am trying to transform a JSON using Jolt transformation looking for some input here. I am trying to get all the items from all the levels into an array.

My goal is to get an array that contains all the items without knowing how many levels I have in the json.

Here is my input and expected output:

if I've 3 levels:

Input:

{
  "id": 11,
  "item": [
    {
      "id": "11_1",
      "item": [
        {
          "id": "11_1_1",
          "item": [
            {
              "id": "11_1_1_1"
            }
          ]
        },
        {
          "id": "11_1_2",
          "item": [
            {
              "id": "11_1_2_1"
            }
          ]
        }
      ]
    }
  ]
}

Expected output:

[
  {
    "id": "11_1"
  },
  {
    "id": "11_1_1"
  },
    {
    "id": "11_1_1_1"
  },
  {
    "id": "11_1_2"
  },
    {
    "id": "11_1_2_1"
  }
]

if I've 2 levels:

Input:

{
  "id": 11,
  "item": [
    {
      "id": "11_1",
      "item": [
        {
          "id": "11_1_1"
        },
        {
          "id": "11_1_2"
        }
      ]
    }
  ]
}

Expected output:

[
  {
    "id": "11_1"
  },
  {
    "id": "11_1_1"
  },
  {
    "id": "11_1_2"
  }
]

I tried to write something like:

[
  {
    "operation": "shift",
    "spec": {
      "item": {   //to cover the second level
        "*": "item"
      }
    }
    },
  {
    "operation": "shift",
    "spec": {
      "item": {
        "*": {    //to cover the 3td level
          "item": {
            "*": "item"
          }
        }
      }
    }
    }
]

The result was null, if I run each transformation separately, I get results when applicable Can you please help me to write a simple spec that will do this?

CodePudding user response：

If the inputs have at most 3 levels as in your case, then use this spec

[
  {
    "operation": "shift",
    "spec": {
      "item": {
        "*": {
          "id": "&",
          "item": {
            "*": {
              "id": "&",
              "item": {
                "*": {
                  "id": "&"
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "id": {
        "*": "[#1].&1"
      }
    }
  }
]

in order to handle the both cases using only this one. If one more level is needed as well, then add

,
"item": {
  "*": {
    "id": "&"
  }
}

just after the inner most

"id": "&"

Edit : If you have some other attributes than id as lately commented like in the below input sample

{
  "id": 11,
  "item": [
    {
      "id": "11_1",
      "quantity": 1,
      "action": "add",
      "state": "x",
      "item": [
        {
          "id": "11_1_1",
          "quantity": 2,
          "action": "drop",
          "state": "y"
        },
        {
          "id": "11_1_2",
          "quantity": 3,
          "action": "modify",
          "state": "z"
        }
      ]
    }
  ]
}

than just a little modification would handle your new case :

[
  {
    "operation": "shift",
    "spec": {
      "item": {
        "*": {
          "*": "&",
          "item": {
            "*": {
              "*": "&",
              "item": {
                "*": {
                  "*": "&"
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": "[#1].&1"
      }
    }
  }
]

which would generate :

[
  {
    "id": "11_1",
    "quantity": 1,
    "action": "add",
    "state": "x"
  },
  {
    "id": "11_1_1",
    "quantity": 2,
    "action": "drop",
    "state": "y"
  },
  {
    "id": "11_1_2",
    "quantity": 3,
    "action": "modify",
    "state": "z"
  }
]

CodePudding user response：

Solution for unlimited levels using library Josson.

https://github.com/octomix/josson

Deserialization

Josson josson = Josson.fromJsonString(
    "{"  
    "  \"id\": 11,"  
    "  \"item\": ["  
    "    {"  
    "      \"id\": \"11_1\","  
    "      \"quantity\": 1,"  
    "      \"action\": \"add\","  
    "      \"state\": \"x\","  
    "      \"item\": ["  
    "        {"  
    "          \"id\": \"11_1_1\","  
    "          \"quantity\": 2,"  
    "          \"action\": \"drop\","  
    "          \"state\": \"y\""  
    "        },"  
    "        {"  
    "          \"id\": \"11_1_2\","  
    "          \"quantity\": 3,"  
    "          \"action\": \"modify\","  
    "          \"state\": \"z\""  
    "        }"  
    "      ]"  
    "    }"  
    "  ]"  
    "}");

Transformation

JsonNode node = josson.getNode("item.cumulateCollect(field(item:), item)");
System.out.println(node.toPrettyString());

Statement field(item:) removes field item from the current object.
The 2nd argument of cumulateCollect() means the next level. In this case is item.

Output

[ {
  "id" : "11_1",
  "quantity" : 1,
  "action" : "add",
  "state" : "x"
}, {
  "id" : "11_1_1",
  "quantity" : 2,
  "action" : "drop",
  "state" : "y"
}, {
  "id" : "11_1_2",
  "quantity" : 3,
  "action" : "modify",
  "state" : "z"
} ]