Decimal Compare is not returning expected values-CodePudding

I am trying to compare two decimals. I am trying to determine if the passed in quantity has too many significant digits. Each item in my database has the number of decimal places accepted for a quantity. So if item 'x' only allows 2 decimal places and I pass in a quantity of 0.00001 it should fail or return non zero.

Decplqty value is 3, for item 'x' (i.e. 3 means only two decimal places. A one would mean no decimal values)

The following Compare code is returning 0

if (decimal.Compare(
    Math.Truncate(requestDto.Quantity * ((item.Decplqty - 1) * 10)), 
    requestDto.Quantity * ((item.Decplqty - 1) * 10)) != 0)
{
...
}

Could somebody please point out my mistake?

CodePudding user response：

It seems that you try to shift decimal places. That does obviously not work with e.g. x * (2 * 10) etc., you want x * (10^2)

This might work the way you intended:

var requestDto = new {Quantity = 1.235m};
var item = new {Decplqty = 3};

decimal sh = (decimal)Math.Pow(10, (item.Decplqty - 1));

int result = 
    decimal.Compare(
        Math.Truncate(requestDto.Quantity * sh), 
        requestDto.Quantity * sh);

result is 0 e.g. for 1, 1.2, 1.23
and -1 for 1.235

CodePudding user response：

If Decplqty value is 3, then your code is multiplying requestDto.Quantity with 20.

Thus, allow me to throw a question back at you: What exactly is the point of multiplying requestDto.Quantity with 20?

Anyway, inserting the values from your question, decimal.Compare will not return 0 as you claim, it will return -1 as expected (indicating that the truncated value is less than the non-truncated value).