Home > Blockchain >  How to reuse slice with sync.pool in golang?
How to reuse slice with sync.pool in golang?

Time:07-07

Questions

  1. Why are the pointers different when I reuse the slice from sync.Pool?
  2. Will it overwrite the data of slice at the second time?

Code

type TestStruct struct {
    Name string
}
func TestSliceWithPool(t *testing.T) {
    var slicePool = sync.Pool{
        New: func() interface{} {
            t.Log("i am created")
            s := make([]interface{}, 0)
            return s
        },
    }

    s, _ := slicePool.Get().([]interface{})
    t.Logf("Lenth: %d, Cap: %d, Pointer: %p", len(s), cap(s), s)
    for i := 0; i < 9000; i   {
        st := &TestStruct{Name: "test"}
        s = append(s, st)
    }
    for _, v := range s {
        if value, ok := v.(TestStruct); ok {
            if value.Name != "test" {
                t.Error("u are changed!")
            }
        }
    }
    s = s[:0]
    slicePool.Put(s)

    s2, _ := slicePool.Get().([]interface{})
    t.Logf("Lenth: %d, Cap: %d, Pointer: %p", len(s), cap(s), s)
    for i := 0; i < 8000; i   {
        st := &TestStruct{Name: "test2"}
        s2 = append(s2, st)
    }
    for _, v := range s2 {
        if value, ok := v.(TestStruct); ok {
            if value.Name != "test2" {
                t.Error("u are changed!")
            }
        }
    }
    slicePool.Put(s2)
}

Description

And the result of test is

slice_test.go:63: i am created
slice_test.go:70: Lenth: 0, Cap: 0, Pointer: 0x1019598
slice_test.go:86: Lenth: 0, Cap: 9728, Pointer: 0xc000500000

Usage

    tmpBuf := make([]byte, 0)
    bf := bytes.NewBuffer(tmpBuf)

    config := goavro.OCFConfig{
        W:     bf,
        Codec: goavroCodec,
    }

    ocfWriter, _ := goavro.NewOCFWriter(config)
    dataCache, _ := slicePool.Get().([]interface{})
    err = ocfWriter.Append(dataCache)

Why is it generated only once but the address is different?And Why the cap is 9728??? Is there any problem when I use the same slice like this?

CodePudding user response:

Why is it generated only once but the address is different?

Because in the first for loop you append to it beyond its capacity, which in New is set to zero, and reassign to it the result of append. For details: Why does append() modify the provided slice? (See example)

Is there any problem when I use the same slice like this?

There could be. When you reslice s with s = s[:0], you are resetting the length but not the capacity. The backing array is still the same one from the previous append-and-reassign operation.

So if you append again to s2, the capacity will be enough to not cause reallocation, and you'll end up overwriting the first elements of the backing array:

A demonstrative example:

func TestSliceWithPool(t *testing.T) {
    var slicePool = sync.Pool{
        New: func() interface{} {
            t.Log("Created")
            s := make([]interface{}, 0)
            return s
        },
    }

    s, _ := slicePool.Get().([]interface{})
    for i := 0; i < 10; i   {
        s = append(s, i)
    }
    fmt.Println(s) 
    // ^ output: [0 1 2 3 4 5 6 7 8 9]

    s = s[:0]
    slicePool.Put(s)

    s2, _ := slicePool.Get().([]interface{})
    fmt.Println(s)
    // ^ output: []

    for i := 0; i < 5; i   {
        s2 = append(s2, i*10)
    }
    fmt.Println(s2) 
    // ^ output: [0 10 20 30 40]

    fmt.Println(s2[:10])
    // ^ output: [0 10 20 30 40 5 6 7 8 9]
}

This might be okay, since you now have a slice with extended capacity that doesn't need reallocation on append, but it could also be a memory leak if your application keeps around other slice headers pointing to the same backing array (as in case of s and s2), thus preventing garbage collection of the buffers.

  •  Tags:  
  • go
  • Related