Home > Mobile >  Extract Bash output to a CSV File for Plotting
Extract Bash output to a CSV File for Plotting

Time:09-08

I am trying to measure the time it takes for a Kubernetes object to be deployed in a Kubernetes cluster by using the time utility. I am trying to do that severally with a time sleep to get values for multiple simulation of deployments.

This is the script.

#!/bin/bash

function time_check {
    i=$1
    time kubectl apply -f deploy.yml --dry-run=client
}

for i in {1..3}
do 
    time_check $i &
    sleep 2
done

This is the Output

deployment.apps/nginx-raw created (dry run)

real    0m0.421s
user    0m0.359s
sys     0m0.138s
deployment.apps/nginx-raw created (dry run)

real    0m0.359s
user    0m0.443s
sys     0m0.158s
deployment.apps/nginx-raw created (dry run)

real    0m0.138s
user    0m0.412s
sys     0m0.122s
deployment.apps/nginx-raw created (dry run)

real    1.483s
user    0m0.412s
sys     0m0.122s
deployment.apps/nginx-raw created (dry run)

real    1.456s
user    0m0.234s
sys     0m0.567s
deployment.apps/nginx-raw created (dry run)

real    2.345
user    0m0.435s
sys     0m0.123s

Goal

I want to pipe the output and take the first row of each iteration's real 0m0.421s , Then take the number part 0m0.421s and strip the 0m if it's in seconds or just leave it if it's in minutes like 1.483. Also strip the s at the end

The final results should be output in a CSV file to be plotted. The expected output in CSV

real
0.421
0.359
0.138
1.483
1.456
2.345

Add-on

I will do this for another deployment and plot the two times data in a line graph to see the time it takes for each deployment

CodePudding user response:

You are using the shell builtin command time. If you switch to linux's time command you can control the output and get the just the data you want.

$ /usr/bin/time -f  '%e' sleep 1.5
1.50

see man time for more details

you can take the output and pipe it into grep -v deployment | tr '\n' ',' that will strip the dry run lines, and convert the renaming newlines into commas

$ printf "1\njunk\n2\njunk\n3\n" 
1
junk
2
junk
3
$ printf "1\njunk\n2\njunk\n3\n" | grep -v junk | tr '\n' ','
1,2,3, $

this is a quick and dirty way to slice the date. I'm sure there are other solutions as well.

CodePudding user response:

I just used a randomized sub-second sleep to get the output stream, but the principle should work.

$: for x in 0 1 2 3 4 5 6 7 8 9; do time sleep 0.$RANDOM; done 2>&1 | 
>    awk 'BEGIN{print "real"}/^real/{ print gensub(/^.*m([0-9.] )s/,"\\1",1)}'
real
0.266
0.716
0.847
0.251
0.358
0.236
0.669
0.266
0.308
0.856

Explained a bit with inline comments -

$: for x in 0 1 2 3 4 5 6 7 8 9; do # this is just a dump loop
     time sleep 0.$RANDOM           # create some output 
   done 2>&1 |                      # dup time output stderr -> stdout
>    awk 'BEGIN{print "real"} # the header
          /^real/{ # match for the lines we want (ignore the rest)
             print gensub(/^.*m([0-9.] )s/,"\\1",1) #  just print matched part
          }'
  •  Tags:  
  • bash
  • Related