value printing from child go routine after defer-CodePudding

I am spwanning 5 worker pools from my driver code, and returning errors from worker pools. In my main, i have another go routine (go routine A , added comment on top of that go routine)listening on errors. But by the time data is picked from my error channel, my defer statement is getting executed. But i can still see logs from go routine A .

func ....{
var requests []Req
        err := json.Unmarshal(Data, &requests)
        if err != nil {
            log.WithError(err).Errorf("Invalid data passed for flag type %v", proto.CreateFlagReq_SET_OF.String())
            return err
        }
        f.Manager.TaskChan = make(chan Req, 100)
        f.Manager.ErrorChan = make(chan error, 100)

        for i := 0; i < f.Manager.WorkerCount; i   {
            f.Manager.Wg.Add(1)
           //AddToSetOfcustomers just validates before addigg to redis
            go f.Manager.Work(ctx, f.redisPool.AddToSetOfcustomers, i)
        }

        for _, request := range requests {
            f.Manager.TaskChan <- request
        }
        close(f.Manager.TaskChan)

        var errors error
        **//go routine A**
        go func() {
            for {
                select {
                case err ,ok:= <- f.Manager.ErrorChan:
                    if ok{
                        errors = multierror.Append(errors, err)
                        log.Errorf("got erro1r %v",errors)
                    }else{
                        log.Info("returning")
                        return
                    }

                }
            }
        }()
        f.Manager.Wg.Wait()

        defer log.Errorf("blhgsgh   %v %v",len(f.Manager.ErrorChan),errors)
        return errors
}


func (m *Manager) Work(ctx context.Context, fn func(string, string, string) error, workerNumber int) {
log.Infof("spawnning worker %v", workerNumber)
defer m.Wg.Done()
defer log.Info("done working")
for {
    select {
    case t, ok := <-m.TaskChan:
        if ok {
            err := fn(t.CustomerName, t.CustomerId, t.Feature)
            if err != nil {
                log.Infof("pushing error from %v",workerNumber)
                m.ErrorChan <- err
            }
        } else {
            return
        }
    case <-ctx.Done():
        log.Infof("closing channel %v", ctx.Err())
        return
    }
}

}

And my logs go like this

info spawnning worker 0
2022/03/14 01:51:44 info spawnning worker 2
2022/03/14 01:51:44 info spawnning worker 1
2022/03/14 01:51:44 info done working
2022/03/14 01:51:44 info done working
2022/03/14 01:51:44 info spawnning worker 3
2022/03/14 01:51:44 info done working
2022/03/14 01:51:44 info spawnning worker 4
2022/03/14 01:51:44 info done working
2022/03/14 01:51:44 info pushing error from 0
2022/03/14 01:51:44 info done working
2022/03/14 01:51:44 error blhgsgh 0
2022/03/14 01:51:44 error got erro1r 1 error occurred: * myError

I kind of suspect defer got executed, then my main go routine finished and then return got executed, but what i can i do to propagate the error i am appending from multiErrors before returning?

if i try to synchronise go routine A and my main go routine(the one where i call defer) using channels, it becomes blocking.Help appretiated

CodePudding user response：

I have simplified your code in the playground.

You appear to be assuming that when f.Manager.Wg.Wait() returns all errors will have been processed. However the errors are being processed in a separate goroutine (**//go routine A**) and you are not waiting for that to complete -- in fact because you do not close f.Manager.ErrorChan the goroutine never completes.

The simplest way to resolve this is to wait for the goroutine to exit before returning from the function. The below example (playground) uses a channel to do this but you could also use a WaitGroup if you prefer.

var errors []error
errDone := make(chan struct{})
go func() {
    for {
        select {
        case err, ok := <-errorChan:
            if ok {
                errors = append(errors, err)
                log.Printf("got error %v", errors)
            } else {
                log.Printf("returning")
                close(errDone)
                return
            }
        }
    }
}()
wg.Wait()

// Everything sending to errorChan is now done so we can safely close the channel
close(errorChan)
<-errDone // Wait for error handling goroutine to complete

Note that defer runs "immediately before the surrounding function returns". Any goroutines that you have started can outlive the function (they are not automatically stopped).