bash script to sum number of "sum" variable and display it with the topic name in a JSON-CodePudding

I have this json that is returned from this command: ./kafka-log-dirs.sh --bootstrap-server localhost:9092 --describe

 {
  "version": 1,
  "brokers": [
    {
      "broker": 0,
      "logDirs": [
        {
          "logDir": "/var/lib/kafka/data/kafka-log0",
          "error": null,
          "partitions": [
            {
              "partition": "pfxdata-mutation-5",
              "size": 1326989,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "__consumer_offsets-13",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "events.oxf-5",
              "size": 279158658,
              "offsetLag": 0,
              "isFuture": false
            }
          ]
        }
      ]
    },
    {
      "broker": 1,
      "logDirs": [
        {
          "logDir": "/var/lib/kafka/data/kafka-log1",
          "error": null,
          "partitions": [
            {
              "partition": "__consumer_offsets-46",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            },
            {
              "partition": "events.oxfcs-10",
              "size": 0,
              "offsetLag": 0,
              "isFuture": false
            }
          ]
        }
      ]
    }
  ]
}

I would like to do a command or bash script to display something like this:

broker 0
Topics                  Size
pfxdata-mutation-5    1326989
__consumer_offsets-13    0
events.oxf-5          279158658
total size            280485647

broker1
Topics                  Size
__consumer_offsets-46    0
events.oxfcs-10          0 
total size               0

so display the partition with the size and then sum all the partitions and then display it on the termianal. Im very new on bash scripting so I dont know how I would start doing this. Thank you

CodePudding user response：

Try this script:

#!/bin/bash
temp=$(jq length sample.json)
len=$((temp-1))

x=0

while [ $x -le $len ]
        do
                partitions=$(jq ".brokers[${x}].logDirs[].partitions[]" sample.json)
                part_size=$(jq ".brokers[${x}].logDirs[].partitions" sample.json)
                total_size=$(echo $part_size | jq 'map(.size) | add')

                echo "broker ${x}"
                echo 'Topics    Size'
                echo $partitions | jq -r '"\(.partition)\t\(.size)"'
                echo "total size        ${total_size}"
                printf "\n"

                x=$(( $x   1 ))
        done

Replace sample.json file using your file

CodePudding user response：

This is an unadorned variant with only jq:

jq -r '.brokers[]
       |.broker,
        (.logDirs[].partitions as $parts
                                |($parts[]|[.partition,.size]),
                                 ["total size",([$parts[].size]|add)]
        |@tsv
        )' sample.json