I have a list of values in a text file like so:

service-1 | a /
service-1 | b /path2/
service-2 | b /path3/
service-2 | b /path4/

I also have a YAML for each of the services like service-1.yaml, service-2.yaml, etc. Each one has an ingress path where I need to put the path:

ingress: 
  instances: 
    - auth: a
      path: 
        <to be filled in>
        <to be filled in>
    - auth: b
      path: 
        <to be filled in>
        <to be filled in>

I want to parse the text file in Shell and then update the appropriate fields in the YAML. I know I can parse the first and second parts using cut, e.g.,

echo "service-1 | a /" | cut -d "|" -f1 or echo "service-1 | a /" | cut -d "|" -f2, but how can I determine which service is which and which paths go where in the YAML? Any suggestions are appreciated.

CodePudding user response：

You can use the YAML processor kislyuk/yq to solve this task. The following code shows a solution for just one file. You can put it in a bash loop that iterates over all files.

SERVICE='service-1'   # or 'service-2'
PATHS='servicePaths.txt'
FILE_IN="$SERVICE.yaml"
FILE_OUT="$SERVICE.out.yaml"

yq -y --arg service $SERVICE --rawfile paths $PATHS '
def getPath($auth):
  $paths / "\n"                                         # split lines by "\n"
  | map(. / "|"                                         # split each line by "|"
        | map(sub("^\\s ";"") | sub("\\s $";""))        # trim strings
        | select(any)                                   # remove empty array (empty lines in file servicePaths.txt)
        | [.[0], (.[1] / " " | .[0], .[1])]             # split auth/path by " "
        | select(.[0] == $service and .[1] == $auth))   # keep only definitions where $service and $auth match
  | .[0][2] // "path undefined";                        # return path of first match or default if no match found

.ingress.instances |= map(.path = getPath(.auth))' "$FILE_IN" > "$FILE_OUT"

File service-1.out.yaml

ingress:
  instances:
    - auth: a
      path: /
    - auth: b
      path: /path2/

File service-2.out.yaml

ingress:
  instances:
    - auth: a
      path: path undefined
    - auth: b
      path: /path3/

Remarks

• yq offers an option -i for inplace editing of files, if you want to replace the original template file instead of using $FILE_OUT.
• the function getPath($auth) builds up a lookup for the paths from the file servicePaths.txt and then selects the correct path for $service and $auth
• .ingress.instances |= map(...) in the last line updates the paths of all services (|= is the update operator)
• the solution is fail safe:
  • if no path for service, auth is defined in servicePaths.txt then "path undefined" is inserted.
  • if more than one path is defined, the first path from servicePaths.txt is inserted.

Variation

If it is valid to have multiple definitions of paths for the same service/auth, use this little modification:

SERVICE='service-1'   # or 'service-2'
PATHS='servicePaths.txt'
FILE_IN="$SERVICE.yaml"
FILE_OUT="$SERVICE.out.yaml"

yq -y --arg service $SERVICE --rawfile paths $PATHS '
def getPath($auth):
  $paths / "\n"                                         # split lines by "\n"
  | map(. / "|"                                         # split each line by "|"
        | map(sub("^\\s ";"") | sub("\\s $";""))        # trim strings
        | select(any)                                   # remove empty array (empty lines in file servicePaths.txt)
        | [.[0], (.[1] / " " | .[0], .[1])]             # split auth/path by " "
        | select(.[0] == $service and .[1] == $auth))   # keep only definitions where $service and $auth match
  | map(.[2]);                                          # return paths of all matches

.ingress.instances |= map(.path = getPath(.auth))' "$FILE_IN" > "$FILE_OUT"

File service-1.out.yaml

ingress:
  instances:
    - auth: a
      path:
        - /
    - auth: b
      path:
        - /path2/

File service-2.out.yaml

ingress:
  instances:
    - auth: a
      path: []
    - auth: b
      path:
        - /path3/
        - /path4/