How to get values in a line while looping line by line in a file (shell script)-CodePudding

I have a file which looks like this (file.txt)

{"key":"AJGUIGIDH568","rule":squid:111-some_random_text_here
{"key":"TJHJHJHDH568","rule":squid:111-some_random_text_here
{"key":"YUUUIGIDH566","rule":squid:111-some_random_text_here
{"key":"HJHHIGIDH568","rule":squid:111-some_random_text_here
{"key":"ATYUGUIDH556","rule":squid:111-some_random_text_here
{"key":"QfgUIGIDH568","rule":squid:111-some_random_text_here

I want to loop trough this line by line an extract the key values.

so the result should be like ,

AJGUIGIDH568
AJGUIGIDH568
YUUUIGIDH566
HJHHIGIDH568
ATYUGUIDH556
QfgUIGIDH568

So I wrote a code like this to loop line by line and extract the value between {"key":" and ","rule": because key values is in between these 2 patterns.

while read p; do 
  echo $p | sed -n "/{"key":"/,/","rule":,/p"
done < file.txt

But this is not working. can someone help me to figure out me this. Thanks in advance.

CodePudding user response：

For the shown data, you can try this awk:

awk -F '"[:,]"' '{print $2}' file

AJGUIGIDH568
TJHJHJHDH568
YUUUIGIDH566
HJHHIGIDH568
ATYUGUIDH556
QfgUIGIDH568

CodePudding user response：

Using sed

$ sed -E 's/([^"]*"){3}([^"]*).*/\2/' input_file
AJGUIGIDH568
TJHJHJHDH568
YUUUIGIDH566
HJHHIGIDH568
ATYUGUIDH556
QfgUIGIDH568

CodePudding user response：

With the give example you can simple use

cut -d'"' -f4 file.txt

CodePudding user response：

Your sample input is almost valid json. You could tweak it to make it valid and then extract the values with jq with something like:

sed -e 's/squid/"squid/' -e 's/$/"}/' file.txt | jq -r .key

Or, if your actual input really is valid json, then just use jq:

jq -r .key file.txt

If the "random-txt" may include double quotes, making it difficult to massage the input to make it valid json, perhaps you want something like:

awk '{print $4}' FS='"' file.txt

sed -n '/{"key":"\([^"]*\).*/s//\1/p' file.txt

while IFS=\" read open_brace key colon val _; do echo "$val"; done < file.txt

CodePudding user response：

Assumptions:

there may be other lines in the file so we need to focus on just the lines with "key" and "rule"
the only text between "key" and "rule" is the desired string (eg, squid never shows up between the two patterns of interest)

Adding some additional lines:

$ cat file.txt
{"key":"AJGUIGIDH568","rule":squid:111-some_random_text_here
ignore this line}
{"key":"TJHJHJHDH568","rule":squid:111-some_random_text_here
ignore this line}
{"key":"YUUUIGIDH566","rule":squid:111-some_random_text_here
ignore this line}
{"key":"HJHHIGIDH568","rule":squid:111-some_random_text_here
ignore this line}
{"key":"ATYUGUIDH556","rule":squid:111-some_random_text_here
ignore this line}
{"key":"QfgUIGIDH568","rule":squid:111-some_random_text_here
ignore this line}

One sed idea:

$ sed -nE 's/^(.*"key":")([^"]*)(","rule".*)$/\2/p' file.txt
AJGUIGIDH568
TJHJHJHDH568
YUUUIGIDH566
HJHHIGIDH568
ATYUGUIDH556
QfgUIGIDH568

Where:

-E - enable extended regex support (and capture groups without need to escape sequences)
-n - suppress printing of pattern space
^(.*"key":") - [1st capture group] everything from start of line up to and including "key":"
([^"]*) - [2nd capture group] everything that is not a double quote (")
(","rule".*)$ - [3rd capture group] everything from ",rule" to end of line
\2/p - replace the line with the contents of the 2nd capture group and print