Running this calculation adds an extra decimal point-CodePudding

I'm running the following calculation and for some reason, an extra decimal is added to the result:

decimal bal = 10.0000
decimal totAmount = 15.0000
decimal payAmount = 3

tempAmount = Math.Ceiling((decimal)(bal / totAmount) * payAmount * 100);
tempAmount = tempAmount / 100;

// tempAmount results in 2.01 as opposed to 2.00

Am I running this incorrectly?

CodePudding user response：

Am I running this incorrectly?

No, but you're introducing an intermediate value (10m/15m) which can't be precisely represented as a decimal. It's being rounded up to 0.6666666666666666666666666667, which then leads to the problem.

If you perform the division as the last operation, you can avoid that, at least in this case:

using System;

decimal bal = 10.0000m;
decimal totAmount = 15.0000m;
decimal payAmount = 3;

decimal tempAmount = Math.Ceiling((bal * payAmount * 100m) / totAmount);
tempAmount = tempAmount / 100;

Console.WriteLine(tempAmount);

That prints out 2 instead of 2.01.

CodePudding user response：

Well, let's have a look at the computations

decimal bal = 10.0000m;
decimal totAmount = 15.0000m;
decimal payAmount = 3m;

decimal tempAmount = Math.Ceiling((decimal)(bal / totAmount) * payAmount * 100);

step by step:

bal / totAmount                                            == 
   0.6666666666666666666666666667m (note the last 7 due to rounding)

(decimal)(bal / totAmount) * payAmount                     == 
   2.0000000000000000000000000001m (note the very last 1)

(decimal)(bal / totAmount) * payAmount * 100               == 
   200.00000000000000000000000001m (note the very last 1)

Math.Ceiling((decimal)(bal / totAmount) * payAmount * 100) == 
  201m

Finally, when you divide 201 by 100 you'll get 2.01