When I am trying to do base64 encode and decode for large files using golang, I am getting byte length missmatched between original and decoded file.

During my testing text file mismatched(1 byte) new line, and with binary file mismatched (2 bytes).

What could be cause missing of these bytes?

package main

import (
    "encoding/base64"
    "io"
    "os"
    "log"
)

func Encode(infile, outfile string) error {
    input, err := os.Open(infile)
    if err != nil {
        return err
    }
    // Close input file
    defer input.Close()

    // Open output file
    output, err := os.Create(outfile)
    if err != nil {
        return err
    }
    // Close output file
    defer output.Close()

    encoder := base64.NewEncoder(base64.StdEncoding, output)
    l, err := io.Copy(encoder, input)
    if err!=nil {
        log.Printf("Failed to encode file:%v",err)
        return err
    } else {
        log.Printf("Wrote %v bytes",l)
    }

    return nil
}

func Decode(infile, outfile string) error {
    input, err := os.Open(infile)
    if err != nil {
        return err
    }
    // Close input file
    defer input.Close()

    // Open output file
    output, err := os.Create(outfile)
    if err != nil {
        return err
    }
    // Close output file
    defer output.Close()

    decoder := base64.NewDecoder(base64.StdEncoding, input)
    l, err := io.Copy(output, decoder)
    if err!=nil {
        log.Printf("Failed to encode file:%v",err)
        return err
    } else {
        log.Printf("Wrote %v bytes",l)
    }

    return nil
}

CodePudding user response：

you don't Close() the encoder so it doesn't flush all the data. from the docs (emphasis mine):

func NewEncoder(enc *Encoding, w io.Writer) io.WriteCloser

NewEncoder returns a new base64 stream encoder. Data written to the returned writer will be encoded using enc and then written to w. Base64 encodings operate in 4-byte blocks; when finished writing, the caller must Close the returned encoder to flush any partially written blocks.

I also quote the example from the docs which has a nice comment:

package main

import (
    "encoding/base64"
    "os"
)

func main() {
    input := []byte("foo\x00bar")
    encoder := base64.NewEncoder(base64.StdEncoding, os.Stdout)
    encoder.Write(input)
    // Must close the encoder when finished to flush any partial blocks.
    // If you comment out the following line, the last partial block "r"
    // won't be encoded.
    encoder.Close()
}