combining numbers from multiple text files using bash-CodePudding

I'm strugling to combine some data from my txt files generated in my jenkins job.

on each of the files there is 1 line, this is how each file look:

testsuite name="mytest" cars="201" users="0" bus="0" bike="0" time="116.103016"

What I manage to do for now is to extract the numbers for each txt file:

awk '/<testsuite name=/{print $3, $4, $5, $6}' my-output*.txt

Result are :

cars="193" users="2" bus="0" bike="0"
cars="23" users="2" bus="10" bike="7"
cars="124" users="2" bus="5" bike="0"
cars="124" users="2" bus="0" bike="123"

now I have a random number of files like this:

my-output1.txt
my-output2.txt
my-output7.txt
my-output*.txt

I would like to create single command just like the one I did above and to sum all of the files to have the following echo result:

cars=544 users=32 bus=12 bike=44

is there a way to do that? with a single line of command?

CodePudding user response：

found a way to do so a bit long:

awk '/<testsuite name=/{print $3, $4, $5, $6}' my-output*.xml | sed -e 's/[^0-9]/ /g' -e 's/^ *//g' -e 's/ *$//g' | tr -s ' ' | awk '{bus =$1;users =$2;cars =$3;bike =$4 }END{print "bus=" bus " users="users " cars=" cars " bike=" bike}'

CodePudding user response：

You can try rquery to do such query.

[ rquery]$ echo 'testsuite name="mytest" cars="201" users="0" bus="0" bike="0" time="116.103016"' > files/output1.txt
[ rquery]$ echo 'testsuite name="mytest" cars="201" users="1" bus="1" bike="2" time="116.103016"' > files/output2.txt
[ rquery]$ echo 'testsuite name="mytest" cars="301" users="10" bus="21" bike="23" time="116.103016"' > files/output3.txt
[ rquery]$ ./rq -q "p d/ /|s 'cars=' sum(trim(substr(@3,strlen('cars=')),'\"')),'users=' sum(trim(substr(@4,strlen('users=')),'\"')),'bus=' sum(trim(substr(@5,strlen('bus=')),'\"')),'bikes=' sum(trim(substr(@6,strlen('bikes=')),'\"'))" files/
cars=703        users=11        bus=22  bikes=25

Check out the latest rquery from here https://github.com/fuyuncat/rquery/releases

CodePudding user response：

With your shown samples please try following awk code, using match function in here. Since awk could read multiple files within a single program itself and your files have .txt format you can pass as .txt format to awk program itself.

awk '
match($0,/[[:space:]] cars="[^"]*" users="[^"]*" bus="[^"]*" bike="[^"]*"/){
  print substr($0,RSTART 1,RLENGTH-1)
}
' Input_file