Home > database >  Javascript >= comparison with two negative numbers producing incorrect result
Javascript >= comparison with two negative numbers producing incorrect result

Time:05-19

I've come across strange behaviour where both before and after using toFixed() the same number is visibly greater than the other (correct). However the toFixed() version produces an incorrect comparison? This seems to happen when both numbers are negative and >= is the operand.

let lastVal = 0.091;
let currentVal = 0.093;
let newVal = 0.094;

let oldSlope = lastVal - currentVal;
let newSlope = currentVal - newVal;
let oldSlopeFixed = (lastVal - currentVal).toFixed(16);
let newSlopeFixed = (currentVal - newVal).toFixed(16);

document.getElementById("one").innerHTML = "oldSlope = "   oldSlope   ", newSlope = "   newSlope;
document.getElementById("two").innerHTML = "oldSlopeFixed = "   oldSlopeFixed   ", newSlopeFixed = "   newSlopeFixed;

if ((newSlope) >= (oldSlope))
{
    document.getElementById("three").innerHTML = "newSlope >= oldSlope  (correct)";
}
if (!((newSlopeFixed) >= (oldSlopeFixed)))
{
    document.getElementById("four").innerHTML = "newSlopeFixed < oldSlopeFixed  (incorrect ... what???)";
}
<!DOCTYPE html>
<html>
<body>

<p id="one"></p>
<p id="two"></p>
<p id="three"></p>
<p id="four"></p>

</body>
</html>

CodePudding user response:

toFixed produces a string. You aren't comparing numbers, you're comparing strings, which are compared alphabetically.

console.log(-0.001 <= -0.002)     // false
console.log('-0.001' <= '-0.002') // true

Don't use toFixed before calculations. It is used when all calculations are done and you're trying to format a number for human consumption.

  • Related