How to read yaml file into bash associative array?-CodePudding

I want to read into bash associative array the content of one yaml file, which is a simple key value mapping.

Example map.yaml

---

a: "2"
b: "3"
api_key: "somekey:thatcancontainany@chara$$ter"

the key can contain any characters excluding space the value can contain any characters without limitations $!:=@etc

What will always be constant is the separator between key and value is :

the script proceses.sh

#!/usr/bin/env bash

declare -A map
# how to read here into map variable, from map.yml file
#map=populatesomehowfrommap.yaml

for key in "${!map[@]}"
do
  echo "key  : $key"
  echo "value: ${map[$key]}"
done

I tried to play around with yq tool, similar to json tool jq but did not have success yet.

CodePudding user response：

With the following limitations:

simple YAML key: "value" in single lines
keys cannot contain :
values are always wrapped in "

#!/usr/bin/env bash

declare -A map
regex='^([^:] ):[[:space:]] "(.*)"[[:space:]]*$'

while IFS='' read -r line
do
    if [[ $line =~ $regex ]]
    then
        printf -v map["${BASH_REMATCH[1]}"] '%b' "${BASH_REMATCH[2]}"
    else
        echo "skipping: $line" 1>&2
    fi
done < map.yaml

Update

Here's a robust solution using yq, which would be simpler if the builtin @tsv filter implemented the lossless TSV escaping rules instead of the CSV ones.

#!/usr/bin/env bash

declare -A map

while IFS=$'\t' read key value
do
    printf -v map["$key"] '%b' "$value"
done < <(
    yq e '
        to_entries | .[] |
        [
            (.key   | sub("\\","\\") | sub("\n","\n") | sub("\r","\r") | sub("\t","\t")),
            (.value | sub("\\","\\") | sub("\n","\n") | sub("\r","\r") | sub("\t","\t"))
        ] |
        join("  ")
    ' map.yaml
)

^{note: the join needs a literal Tab}

CodePudding user response：

One way, is by letting yq output each key/value pair on a single line, in the following syntax:

key@value

Then we can use bash's IFS to split those values.
_{The @ is just an example and can be replaced with any single char}

This works, but please note the following limitations:

It does not expect nested values, only a flat list`
The field seperator (@ in the example) does not exist in the YAML key/value's

#!/bin/bash

declare -A arr
while IFS="@" read -r key value
do
    arr[$key]="$value"
done < <(yq e 'to_entries | .[] | (.key   "@"   .value)' input.yaml)

for key in "${!arr[@]}"
do
    echo "key  : $key"
    echo "value: ${arr[$key]}"
done

$ cat input.yaml
---
a: "bar"
b: "foo"

$
$
$ ./script.sh
key  : a
value: bar
key  : b
value: foo
$

CodePudding user response：

I used @Fravadona s answer so will mark it as answer

After some modification to my use case, what worked for me looks like:

DEFS_PATH="definitions"

declare -A ssmMap
for file in ${DEFS_PATH}/*
do
    filename=$(basename -- "$file")
    projectName="${filename%.*}"

    regex='^([^:] ):[[:space:]]*"(.*)"[[:space:]]*$'
    while IFS='' read -r line
    do
        if [[ $line =~ $regex ]]
        then
            value="${BASH_REMATCH[2]}"
            value=${value//"{{ ssm_env }}"/$INFRA_ENV}
            value=${value//"{{ ssm_reg }}"/$SSM_REGION}
            value=${value//"{{ projectName }}"/$projectName}
            printf -v ssmMap["${BASH_REMATCH[1]}"] '%b' "$value"
        else
            echo "skipping: $line" 1>&2
        fi
    done < "$file"
done

Basically in real use case I have one folder where yaml definitions are located. I iterate over all of them to form the associative array ssmMap