I have a .congifg file with the following content.

train_input_reader: {
  tf_record_input_reader {
    input_path: ["train.record-001", 
                 "train.record-002",
                 "train.record-003"]
  }
}

eval_input_reader: {
  tf_record_input_reader {
    input_path: ["test.record-0001"]
  }
}

I want to replace the input_path of train_input_reader with a new file path using a shell script,the content of the file list (train_new.record-001 & train_new.record-002) returned by ls, there is my attempt below.

datalist=`ls ${dataset_path}train_new.record-*`
sed -i.'org' "/train_input_reader:/n;n;s|^ *input_path:.*|    input_path: [\"$datalist\"]| " ${folder}${config_path}

result:

train_input_reader: {
  tf_record_input_reader {
    input_path: ["train_new.record-001, train_new.record-002, "]
                 "train.record-002",
                 "train.record-003"]
  }
}
.....

This is not what I expected, I expected that I would get the following results

train_input_reader: {
  tf_record_input_reader {
    input_path: ["train_new.record-001",
                 "train_new.record-002"]
  }
}
.....

What should I do? This is my first time using sh.

CodePudding user response：

Would you please try the following:

#!/bin/bash

files=( "$dataset_path"train_new.record-* )     # assign an array "files" to the list of matched files
datalist="\"${files[0]##*/}\""                  # 1st element of the list, dirname removed and enclosed by double quotes
for (( i = 1; i < ${#files[@]}; i   )); do      # append remaining elements by prepending comma, newline and tabs
    datalist =$',\\n\t\t"'"${files[$i]##*/}\""
done

sed -i'.org' -E '
:l                                              ;# define a label "l"
N                                               ;# append next line
$!b l                                           ;# go to label "l" unless eof
# now the pattern space holds the entire file
# then perform the replacement across the lines
s|(train_input_reader:[^}]*input_path: *\[)[^]]*|\1'"$datalist"'|' "$folder$config_path"

Output:

train_input_reader: {
  tf_record_input_reader {
    input_path: ["train_new.record-001",
                "train_new.record-002"]
  }
}

eval_input_reader: {
  tf_record_input_reader {
    input_path: ["test.record-0001"]
  }
}

As you mention you're working with MacOS, it is tested with bash 3.2 on Linux. But I'm not confident about the interoperability of sed.

[Edit]
Explanation about the regex:

• As (train_input_reader:[^}]*input_path: *\[) is enclosed with parentheses, the matched substring is reused in the replacement text as \1.
• [^}]* matches any characters other than }. It prevents the longest match to the last input_path.
• [^]]* matches zero or more sequence of any characters other than ]. It will match the value of input_path enclosed with square brackets and will be replaced with datalist.