Here's my code (writeFromProcessToFileWithMax is an internal function, and is working properly):

    // Go routines for non-blocking reading of stdout and stderr and writing to files
    g := new(errgroup.Group)

    // Read stdout in goroutine.
    g.Go(func() error {
        err = writeFromProcessToFileWithMax(stdoutScanner, stdoutFileWriter, maxStdoutFileLengthInGB)
        if err != nil {
            log.Error().Err(err).Msgf("Error writing to stdout file: %s", stdoutFilename)
            return err
        }
        return nil
    })

    // Read stderr in goroutine.
    g.Go(func() error {
        err = writeFromProcessToFileWithMax(stderrScanner, stderrFileWriter, maxStderrFileLengthInGB)
        if err != nil {
            log.Error().Err(err).Msgf("Error writing to stderr file: %s", stderrFilename)
            return err
        }
        return nil
    })

    // Wait the command in a goroutine.
    g.Go(func() error {
        return cmd.Wait()
    })

    // Starting the command
    if err = cmd.Start(); err != nil {
        log.Error().Err(err).Msg("Error starting command")
        return err
    }

    // Waiting until errorGroups groups are done
    if err = g.Wait(); err != nil {
        log.Error().Err(err).Msg("Error during running of the command")
    }

When I run it, I get the following Error = Error during running of the command error="exec: not started". But everything works properly.

Will the come back to bite me or should I suppress?

CodePudding user response：

You're waiting for cmd before you start it. cmd.Wait() would be invoked before cmd.Start() most of the time in your old code. (There is no guarantee when exactly exactly things in two different goroutines happen with respect to each other, unless you explicitly use synchronisation points)

Swap the order of cmd.Start() and the cmd.Wait() inside the goroutine:

// Starting the command
if err = cmd.Start(); err != nil {
    log.Error().Err(err).Msg("Error starting command")
    return err
}

// Wait the command in a goroutine.
g.Go(func() error {
    return cmd.Wait()
})

When you start the goroutine that waits after starting the command, you're guaranteed to perform cmd.Start() and cmd.Wait() in the correct order.

As to why it seemed to work: g.Wait() returns after the first failing goroutine in the group (or after all complete successfully). Since the goroutine that did cmd.Wait() failed, g.Wait() returned before the goroutines that wrote the process output to a file completed.

So it didn't really work; the file writing wasn't complete when you expected it to be complete, and if you exited the process at that point, or tried to read the files, you may have seen incomplete data.