Remove substring from a json retrieved in a bash script-CodePudding

I do not have experience working with bash, and I have to update a small script to remove a substring from all the values in a JSON with a common pattern /development/dev/. I get something like this from GetParametersByPath (AWS service):

{
    "Parameters": [
        {
            "Name": "/development/dev/var1",
            "Type": "String",
            "Value": "Saanvi Sarkar",
            "Version": 1
        },
        {
            "Name": "/development/dev/var2",
            "Type": "String",
            "Value": "Zhang Wei",
            "Version": 1
        },
        {
            "Name": "/development/dev/var3",
            "Type": "String",
            "Value": "Alejandro Rosalez",
            "Version": 1
        },
        
    ]
}

I wanna remove the substring "/development/dev/" for all the Name values.

This is what I have at the moment

  // parameter_store_path has the value "/development/dev/"
 jq_actions=$(echo -e ".Parameters | .[] | [.Name, .Value] | \042\(.[0])=\(.[2])\042 | sub(\042${parameter_store_path}/\042; \042\042)")

// function that returns the JSON 
  aws ssm get-parameters-by-path \
  --path $parameter_store_path \
  --with-decryption \
  --region eu-west-2 \
  | jq -r "$jq_actions" >> /opt/elasticbeanstalk/deployment/custom_env_var

  cp /opt/elasticbeanstalk/deployment/custom_env_var /opt/elasticbeanstalk/deployment/env

  #Remove temporary working file.
  rm -f /opt/elasticbeanstalk/deployment/custom_env_var

  #Remove duplicate files upon deployment.
  rm -f /opt/elasticbeanstalk/deployment/*.bak

I was reading some docs about bash, and I saw that I can replace part of the string using ${FOO#prefix}, but I don't know how to implement it in this code.

CodePudding user response：

You already seem to know how to perform substitutions in jq. Just do slightly more.

aws ssm get-parameters-by-path \
  --path "$parameter_store_path" \
  --with-decryption \
  --region eu-west-2 |
jq -r '.Parameters | .[] |
  "\(.Name | sub("/development/dev/"; ""))=\(.Value)"'

(The subscript [2] didn't make sense here - I'm guessing you meant [1]. However, the subscripting seemed superfluous, anyway; I refactored to simplify it away, and apply the substitution only on the Name. Perhaps also add double quotes around the Value?)

Demo: https://jqplay.org/s/o7Exg4Ns6BU

If you want to pass in the path as a variable, try

jq --arg path "/development/dev/" \
   -r '.Parameters | .[] |
  "\(.Name | sub($path; ""))=\"\(.Value)\""'

Demo: https://replit.com/@tripleee/aws-jq-demo#main.sh

This also gets rid of the useless use of echo -e. In this particular case, I simply switched to single quotes instead of double (though notice that they do slightly different things - generally prefer single quotes, unless you need to perform command substitution or variable substitution in your string, or if it needs to contain literal single quotes. Bash also provides $'\047C-style\047 strings' with a backslash interpretation facility similar to the one in echo -e - the example demonstrates how to embed literal single quotes with the octal escape \047).

A parameter expansion only really makes sense when what you have is already a string in a Bash variable.

Here's a refactoring of your script which also avoids the weird cp rm in favor of a simple mv. I'm guessing you are writing something else to the target file earlier in the script and that's why you append these values from jq.

#!/bin/bash
parameter_store_path="/development/dev/"

aws ssm get-parameters-by-path \
  --path "$parameter_store_path" \
  --with-decryption \
  --region eu-west-2 |
jq --arg path "$parameter_store_path" \
   -r '.Parameters | .[] |
  "\(.Name | sub($path; ""))=\"\(.Value)\""
  ' >> /opt/elasticbeanstalk/deployment/custom_env_var

mv /opt/elasticbeanstalk/deployment/custom_env_var /opt/elasticbeanstalk/deployment/env

rm -f /opt/elasticbeanstalk/deployment/*.bak