assign context of a .csv file, line by line as a variable into a linux command script-CodePudding

I have a .csv file that contains 2500 unique request id like "4485-182-65846". I want to run a elasticsearch query command that contain this request Id. my query be like:

curl -XGET 127.0.0.1:9200/_search?pretty -d '

{
     "query": {
             "match": {
                "request_id": "$VARIABLE(contents of the file)"
               }
         }
}' > answer.csv

Now I want to put every unique id into a VARIBALE and run the query to answer them in a specific file

I would appreciate any help.

I tried this code but did not answer

request_id=(cat file.csv)

for i in request_id;

do

curl -XGET 127.0.0.1:9200/_search?pretty -d '

{
     "query": {
             "match": {
                "request_id": "$i"
               }
         }
}' > answer.csv

CodePudding user response：

jq is a good tool for manipulating JSON. For example, assuming the contents of your file.csv are:

4485-182-65846
4485-182-65847
4485-182-65848
4485-182-65849
4485-182-65850

You can generate JSON using these values via:

$ json_template='{ "query": { "match": { "request_id": "$id" }}}'
$ while read -r id ; do req=$(echo $json_template | jq --arg jqid "$id" '.query.match.request_id |= $jqid'); echo "$req" ; done <file.csv

Generated JSON looks like:

{
  "query": {
    "match": {
      "request_id": "4485-182-65846"
    }
  }
}

You could then use the JSON in your HTTP query by changing the while read... loop to:

while read -r id ; do req=$(echo '{ "query": { "match": { "request_id": "$id" }}}' | jq --arg jqid "$id" '.query.match.request_id |= $jqid'); curl -XGET https://127.0.0.1:9200/_search?pretty -d "$req" ; done <file.csv

(Note, your code has additional oddities aside from those identified by @user1934428. Your GET URL has no https protocol.)

CodePudding user response：

Thank you guys for your attention and answers, this worked for me:

VAR=$(cat file1.txt)

for i in $VAR;

do 

curl --user userpasswrd:password -XGET "http://127.0.0.1:9200/_search?pretty" -H 'Content-Type: application/json' -d'

{
     "query": {
             "match": {
                "log.request_id": '$i'
               }
         }
}' >> answer.csv

note: I had to save the file1.txt with UTF8 encoding and put all request id between double quotation