Home > Enterprise >  Why Is uint64 of NaN and MaxFloat64 equal in Golang?
Why Is uint64 of NaN and MaxFloat64 equal in Golang?

Time:12-24

The following condition evaluates to true, looking for an explanation as to why -

x, y := math.NaN(), math.MaxFloat64 

fmt.Println(uint64(x) == uint64(y))

>>> true

Specifically, why is uint64(math.NaN()) = 9223372036854775808?

CodePudding user response:

Language runtimes usually do not pay attention to what happens if you try to convert a NaN value to an integer. They typically simply issue the corresponding hardware instruction, and let the "hardware" do what it does.

So, the question is what does the hardware do in this case? Of course this depends on what your particular machine/ISA is. Assuming the most common Intel x86 architecture, the corresponding instruction used is most likely CVTTSD2SI, though you should check the assembly generated by your program to make sure this is indeed the case. Here's the relevant quote regarding this instruction's behavior:

If a converted result exceeds the range limits of signed quadword integer (in 64-bit mode and REX.W/VEX.W/EVEX.W = 1), the floating-point invalid exception is raised, and if this exception is masked, the indefinite integer value (80000000_00000000H) is returned.

Based on this, what's happening is that the processor raises the invalid exception, which is masked by most language run-times by default, so your program simply keeps continuing. Consequently, you get the value 0x8000000000000000 as specified by the instruction, which is equivalent to 9223372036854775808 in decimal, explaining the result you observed.

If you worry about these conversions, you should either check for valid range before doing the conversion, or unmask the exception and let your program get a floating-point exception. Of course, you should carefully consider if it is worth doing the extra check as it will be a performance hit. Unmasking the exception, while morally the "correct" thing to do, is also problematic, as you wouldn't want your "number-crunching" program to crash at run-time. Obviously, the right thing to do depends on exactly what your circumstances/needs are.

Note:

Here's the assembly output for your program on x86: https://go.godbolt.org/z/YfqKdvY6E You can see the call to cvttsd2si on line 100 and others. Here's a simpler program that compiles to a shorter assembly, whose output is easier to read, and showing the use of cvtsd2si more prominently on line 14: https://go.godbolt.org/z/r7Pq5Tqqh

  • Related