Home > Software engineering >  Removing decimal places after second number
Removing decimal places after second number

Time:02-10

I have some raw data exported from my ERP. Most of the data has two decimals, but there are a few lines contain numbers with three or more decimal places... When summing up all data with powershell I have a 0,01€ discrepancy between my ERP PDF report and sum of exported data. I tracked the discrepancy to a single number - 2.205. Powershell rounds this number to 2.2 not 2.21 as expected. If I try any other number - 2.215, 2.225 they all get rounded to 2.22, 2.23..

Why is 2.205 not treated the same???

CodePudding user response:

.NET's default mid-point rounding strategy is ToEven, meaning that .5 values are rounded to the closest even integer.

Important: The following example use [decimal] numbers - as implied by number-literal suffix d - not [double]s , so as to avoid problems that stem from [double] values, which internally use a binary representation that typically does not have an exact decimal equivalent, so that a number that may appear to have a 5 in the relevant decimal place may actually be just above or below that exact decimal value,[1] which means that mid-point rounding may not apply - see Rounding and precision.

In the case at hand, rounding -2.205 to 2 decimal places means that 0 is the closest even integer, and [Math]::Round(-2.205d, 2) therefore yields -2.20.[2]

It sounds like you want AwayFromZero as the mid-point rounding strategy, which rounds to the next higher integer in absolute terms:

PS> [Math]::Round(-2.205d, 2, 'AwayFromZero')
-2.21

[1] To see if a given fractional value has an exact decimal representation, you can use something like the following:
$num = [double] -2.205; '{0:F28}' -f $num
Unless the input number is echoed with trailing zeros only, the given value has no exact decimal representation; [double] value -2.205 does not, but 0.5 does, for instance.

[2] Note that [Math]::Round(2.225d, 2) therefore yields 2.22, not 2.23, as you state in your question.

CodePudding user response:

If I try any other number - 2.215, 2.225 they all get rounded to 2.22, 2.23

Are you sure about that 2.23 result? Here's my reslts when I run it:

PS H:\> [math]::Round(2.205,2)
2.2
PS H:\> [math]::Round(2.215,2)
2.22
PS H:\> [math]::Round(2.225,2)
2.22
PS H:\> [math]::Round(2.235,2)
2.24
PS H:\> [math]::Round(2.245,2)
2.24

Taken from Scripting guy:

Notice that here the decimal, .25, rounded down to .2. This is because the number (2) is even. In the following example, the .35 rounds up because the number (3) is odd.

To fix it, define Midpoint Routing per Microsoft's documentation:

[math]::Round(2.205,2,1)
  • Related