Home > Mobile >  Rounding Behavior When Converting from Double to String in PowerShell 5.1
Rounding Behavior When Converting from Double to String in PowerShell 5.1

Time:11-30

When converting from double to a string using [string], strange rounding behavior seems to occur in PowerShell 5.1.

Converting 0.114338713266919499 to a string becomes 0.11433871326919, indicating that it rounds the ...499 down to zero and truncates. This behavior doesn't seem to match rounding in C# , so I am struggling to duplicate this behavior as the rounding doesn't seem to follow any convention I can find. I found the conversions page for powershell but it has no details on how the conversion works under the hood. I am trying to duplicate the values generated by a script using a double to string conversion in C# and cannot change the script. Any details on the conversion process would be appreciated.

I tried a variety of double values and it seems to keep 15 digits after the decimal not including zeroes after the decimal but before non-zero values. However, when I go to C# and ask it to round the 15th decimal place, I find different results. I understand the floating point numbers are not exact values, but it seems since they are the same range in C# and PowerShell for doubles that I should be able to duplicate the behavior.

CodePudding user response:

It is not PowerShell issue, it is the nature of floating point numbers and also the 15-17 digits limit is the same at .Net. Take a look at here. Generally it's an issue (if we can call it so) with any code that uses floating point numbers, e.g. in any language that uses IEEE 754. Test this code and you will get false!

double myVar = 22.3;
Console.WriteLine((myVar   .1) == 22.4);

It means if you see a double variable with value 22.3 it is not actually that precise value but just an approximation. Just use decimal data type if you are working with precise decimal values.

  • Related