Why does io.Copy sometimes lose lines when reading from a socket?-CodePudding

Please help me understand the observed behavior of reading from a socket.

In the experiment, the sender always sends the same lines, which end in '\n', then closes the write end of a socket.

This code on a receiving side works as expected, printing each line:

    rdr := bufio.NewReader(sock)
    for {
        b, err := rdr.ReadBytes('\n')
        if err != nil {
            break
        }
        fmt.Print(string(b))
    }

However, this code

n, err := io.Copy(os.Stdout, sock)

sometimes skips a random number of lines from the beginning of the data block, and only prints the rest (n changes accordingly, and err is always nil).

The sock is a custom type, which abstracts net.TCPConn and tls.Conn, which is otherwise used throughout the codebase w/o problems.

Why lines at the beginning of the read by io.Copy are sometimes lost?

CodePudding user response：

bufio.Reader implements buffering for an io.Reader object. What does this mean? If you use a bufio.Reader to read from an io.Reader source, it may read more data from its source (and buffer it) than what you read directly from the bufio.Reader itself.

This means if you use a bufio.Reader to read from the sock first, and then you use io.Copy(), there may be some data already read from sock and sitting in the bufio.Reader's internal buffer, which io.Copy() will not see and will not copy.

You shouldn't mix those 2 on the same io.Reader source. If you must, then be sure to first drain the bufio.Reader's buffer, then proceed to copy from sock like this:

// First drain the buffer:
n, err := io.Copy(os.Stdout, rdr)
// Handle error

// Then proceed with the socket:
n, err = io.Copy(os.Stdout, sock)
// Handle error