I'm trying to convert 64bit decimal to hex but I get a wrong result
Public Function DecToBinary(dec As Double) As String
DecToBinary= Hex(dec)
End function
For dec=3689348814742970688 I get 0x3333333333436200 , I should get 0x3333333333436140
I will use the hex to get the binary data cause I couldn't find another way to get a string with the bits.
In the end 3689348814742970688 will become a string with bits "0011001100110011001100110011001100110011010000110110000101000000"
Thanks in advance
CodePudding user response:
Your assumption that dec
contains 3689348814742970688 is wrong.
3689348814742970688 exceeds the precision range of a Double
, so dec
stores the "rounded" value of 3689348814742970880 instead.
To solve this issue, use a BigInteger
from the System.Numerics
namespace instead of a Double
. It's .NET's data type for arbitrarily large integers.
' prints 3333333333436140
Console.WriteLine(BigInteger.Parse("3689348814742970688").ToString("x"))
CodePudding user response:
This might work,
Dim l As Long = 3689348814742970688L
Dim s As String = Convert.ToString(l, 2)
Debug.WriteLine(s.ToString.PadLeft(64, "0"c))
l = Long.MaxValue
s = Convert.ToString(l, 2)
Debug.WriteLine(s.ToString.PadLeft(64, "0"c))