How to measure execution time of concurrent functions?-CodePudding

I have for loop which executes goroutine. Here is the piece of code how it looks:

var wg sync.WaitGroup
for _, photo := range photos {
        wg.Add(1)
        
        go func() {
            defer wg.Done()
            start := time.Now()
            
            result := process(photo)
            
            fmt.Println("took %s", time.Since(start))
        }()
    }
    
    go func() {
        wg.Wait()
    }()

In the above code, I want to count execution time of each process(photo) function. But the way I do it seems wrong. The numbers I get is not real. Where is my mistake and how to fix it?

CodePudding user response：

You need to pass the photo variable to your go-routine or introduce a local variable inside the for loop and pass that variable to process() function, otherwise all your go-routines will have the same instance of photo variable.

And, there is no point writing wg.Wait() in another go-routine. You can updated your code like this

var wg sync.WaitGroup
for _, photo := range photos {
        wg.Add(1)

        photo := photo

        go func() {
            defer wg.Done()
            start := time.Now()
            
            result := process(photo)
            
            fmt.Println("took %s", time.Since(start))
        }()
  }
  wg.Wait()

CodePudding user response：

Instead of creating as many goroutines as you have pictures to process you would be better off creating as many goroutines as you have CPU cores so that there is no wasteful usage or exhaustion of system resources. Hand out one picture to every idle goroutine and make time it while it processes the picture. When picture processing is over and you stopped the timer, log the time and hand that goroutine another picture to process. No goroutines waiting to be scheduled that consume memory.

See https://gobyexample.com/worker-pools for a starting point.