Home > Back-end >  int64(math.Pow(2, 63) - 1) results in -9223372036854775808 rather than 9223372036854775807
int64(math.Pow(2, 63) - 1) results in -9223372036854775808 rather than 9223372036854775807

Time:07-10

I am trying to store max and min signed ints of different bits. The code works just fine for ints other than int64

package main

import (
    "fmt"
    "math"
)

func main() {
    var minInt8 int8 = -128
    var maxInt8 int8 = 127
    fmt.Println("int8\t->", minInt8, "to", maxInt8)
    fmt.Println("int8\t->", math.MinInt8, "to", math.MaxInt8)

    var minInt16 int16 = int16(math.Pow(-2, 15))
    var maxInt16 int16 = int16(math.Pow(2, 15) - 1)
    fmt.Println("int16\t->", minInt16, "to", maxInt16)
    fmt.Println("int16\t->", math.MinInt16, "to", math.MaxInt16)

    var minInt32 int32 = int32(math.Pow(-2, 31))
    var maxInt32 int32 = int32(math.Pow(2, 31) - 1)
    fmt.Println("int32\t->", minInt32, "to", maxInt32)
    fmt.Println("int32\t->", math.MinInt32, "to", math.MaxInt32)

    var minInt64 int64 = int64(math.Pow(-2, 63))
    var maxInt64 int64 = int64(math.Pow(2, 63) - 1) // gives me the wrong output
    fmt.Println("int64\t->", minInt64, "to", maxInt64)
    fmt.Println("int64\t->", math.MinInt64, "to", math.MaxInt64)
}

Output:

int8    -> -128 to 127
int8    -> -128 to 127
int16   -> -32768 to 32767
int16   -> -32768 to 32767
int32   -> -2147483648 to 2147483647
int32   -> -2147483648 to 2147483647
int64   -> -9223372036854775808 to -9223372036854775808
int64   -> -9223372036854775808 to 9223372036854775807

I have no idea about the cause of this behavior, any help would be appreciated.

CodePudding user response:

There are multiple problems here:

math.Pow returns a float64. This type cannot be used to represent a 64 bit signed integer with full precision as required for the attempted computation here. To cite from Double-precision floating-point format

Precision limitations on integer values

  • Integers from −2^53 to 2^53 (−9,007,199,254,740,992 to 9,007,199,254,740,992) can be exactly represented
  • Integers between 2^53 and 2^54 = 18,014,398,509,481,984 round to a multiple of 2 (even number)
  • Integers between 2^54 and 2^55 = 36,028,797,018,963,968 round to a multiple of 4

Even if the precision would be sufficient (which is true in the special case of 2^63) then the precision of float64 is not sufficient to substract 1 from 2^63. Just try the following (uint64 is used here since signed int64 is not sufficient):

uint64(math.Pow(2, 63))    // -> 9223372036854775808
uint64(math.Pow(2, 63)-1)  // -> 9223372036854775808

Converting the value first to uint64 and then subtracting works instead, but only because 2^63 can be represented with full prevision in float64 even though other values with this size can not:

uint64(math.Pow(2, 63))-1  // -> 9223372036854775807
  • Related