I'm working through The Go Programming Language and learning about goroutines, and came across the following issue. In this example, the following function is meant to take a channel of files and process each of them:

func makeThumbnails5(filenames <-chan string) int64 {
    sizes := make(chan int64)
    var wg sync.WaitGroup
    for f := range filenames {
        wg.Add(1)
        // worker
        go func(f string) {
            defer wg.Done()
            thumb, err := thumbnail.ImageFile(f)
            if err != nil {
                log.Println(err)
                return
            }
            info, _ := os.Stat(thumb)
            sizes <- info.Size()
        }(f)
    }

    // closer
    go func() {
        wg.Wait()
        close(sizes)
    }()

    var total int64
    for size := range sizes {
        total  = size
    }

    wg.Wait()
    return total
}

I've tried to use this function the following way:

func main() {
    thumbnails := os.Args[1:] /* Get a list of all the images from the CLI */
    ch := make(chan string, len(thumbnails))
    for _, val := range thumbnails {
        ch <- val
    }
    makeThumbnails5(ch)
}

However, when I run this program, I get the following error:

fatal error: all goroutines are asleep - deadlock!

It doesn't appear that the closer goroutine is running. Could someone help me understand what is going wrong here, and what I can do to run this function correctly?

CodePudding user response：

As I commented it deadlocks because the filenames chan is never closed and thus the for f := range filenames loop never completes. However, just closing the input chan means that all goroutines launched in the loop would get stuck at the line sizes <- info.Size() until the loop ends. Not a problem in this case but if the input can be huge it could be (then you'd probably want to limit the number of concurrent workers too). So it makes sense to have the main loop in a goroutine too so that the for size := range sizes loop can start consuming. Following should work:

func makeThumbnails5(filenames <-chan string) int64 {
    sizes := make(chan int64)
    var wg sync.WaitGroup
    wg.Add(1)
    go func() {
        defer wg.Done()
        for f := range filenames {
            wg.Add(1)
            // worker
            go func(f string) {
                defer wg.Done()
                thumb, err := thumbnail.ImageFile(f)
                if err != nil {
                    log.Println(err)
                    return
                }
                info, _ := os.Stat(thumb)
                sizes <- info.Size()
            }(f)
        }
    }()

    // closer
    go func() {
        wg.Wait()
        close(sizes)
    }()

    var total int64
    for size := range sizes {
        total  = size
    }

    return total
}

The implementation of the main has a similar problem that if the input is huge you're essentially load it all into memory (buffered chan) before passing it on to be processed. Perhaps something like following is better

func main() {
    ch := make(chan string)
    go func(thumbnails []string) {
        defer close(ch)
        for _, val := range thumbnails {
            ch <- val
        }
    }(os.Args[1:])
    makeThumbnails5(ch)
}