How to Iterate Json key value pair and get the value and store in another json using Bash Script wit-CodePudding

Im writing bash script for AWS Secret Manager .

Question:

I have one aws secret manager it will contain multiple secret

Example: aws secretsmanager get-secret-value --secret-id Example | jq -r ".SecretString"

input.json

{"Username":"admin","Password":"admin","Endpoint":"localhost","DatabaseName":"example","Port":"3306"}

Using this command got a json output

Now I want to extract All values like (Username , password,..)

After get all extract values add in another json file called (secondfile.json)

I have another json file it will contain

this is secondfile.json

 {
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Aurora": {
    "Username": "",
    "Password": "",
    "Endpoint": "",
    "DatabaseName": "",
    "Port": ""
  }
}

Sample Output: extracted values now stored second file.json file in Aurora[] array elements.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Aurora": {
    "Username": "admin",
    "Password": "admin",
    "Endpoint": "localhost",
    "DatabaseName": "example",
    "Port": "3306"
  }
}

Need Help on do with bash script .

Note: Input.json value not fixed , it may be contain 3 json key/value or n number of key/value

CodePudding user response：

If I understand your question correctly, you simply want to embed/wrap the input document in a top level document.

echo '{"Username":"admin","Password":"admin","Endpoint":"localhost","DatabaseName":"example","Port":"3306"}' | jq '{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Aurora": .
}'

Output:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Aurora": {
    "Username": "admin",
    "Password": "admin",
    "Endpoint": "localhost",
    "DatabaseName": "example",
    "Port": "3306"
  }
}

The key "Aurora" will contain your input document. If you need only specific fields, then replace . with {Username,Password}

CodePudding user response：

Here is one solution how to jq's --slurpfile option to use another file as "template" to format your input and then replace the parts according to your requirements:

$ cat template.json
{
    "Logging": {
        "LogLevel": {
            "Default": "Information",
            "Microsoft.AspNetCore": "Warning"
        }
    },
    "Aurora": {
        "Username": "",
        "Password": "",
        "Endpoint": "",
        "DatabaseName": "",
        "Port": "",
        "Environment": ""
    }
}

$ cat input.json
{
  "Username": "admin",
  "Password": "admin",
  "Endpoint": "localhost",
  "DatabaseName": "example",
  "Port": "3306",
  "SomethingElse": "ignore me"
}

$ jq --slurpfile tpl template.json '$tpl[0]   {Aurora: with_entries(select(.key | IN($tpl[0].Aurora|keys[])))}' <input.json
{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Aurora": {
    "Username": "admin",
    "Password": "admin",
    "Endpoint": "localhost",
    "DatabaseName": "example",
    "Port": "3306"
  }
}

Or swapping template and input files:

$ jq --slurpfile in input.json '.Aurora |= keys as $keys | $in[0] | with_entries(select(.key | IN($keys[])))' <template.json
{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Aurora": {
    "Username": "admin",
    "Password": "admin",
    "Endpoint": "localhost",
    "DatabaseName": "example",
    "Port": "3306"
  }
}

CodePudding user response：

If you can change your "template" file, I would suggest not keeping it as JSON, but make it a jq program instead:

$ cat input.json
{
  "Username": "admin",
  "Password": "admin",
  "Endpoint": "localhost",
  "DatabaseName": "example",
  "Port": "3306",
  "SomethingElse": "ignore me"
}

$ cat filter.jq
{
    "Logging": {
        "LogLevel": {
            "Default": "Information",
            "Microsoft.AspNetCore": "Warning"
        }
    },
    "Aurora": {
        "Username": .Username,
        "Password": .Password,
        "Endpoint": .Endpoint,
        "DatabaseName": .DatabaseName,
        "Port": .Port
    }
}

$ jq -f filter.jq < input.json
{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "Aurora": {
    "Username": "admin",
    "Password": "admin",
    "Endpoint": "localhost",
    "DatabaseName": "example",
    "Port": "3306"
  }
}

Note that the filter program can be written with shorthand syntax:

$ cat filter.jq
{
    "Logging": {
        "LogLevel": {
            "Default": "Information",
            "Microsoft.AspNetCore": "Warning"
        }
    },
    "Aurora": {
        Username,
        Password,
        Endpoint,
        DatabaseName,
        Port
    }
}