I've come across strange behaviour where both before and after using toFixed() the same number is visibly greater than the other (correct). However the toFixed() version produces an incorrect comparison? This seems to happen when both numbers are negative and >= is the operand.
let lastVal = 0.091;
let currentVal = 0.093;
let newVal = 0.094;
let oldSlope = lastVal - currentVal;
let newSlope = currentVal - newVal;
let oldSlopeFixed = (lastVal - currentVal).toFixed(16);
let newSlopeFixed = (currentVal - newVal).toFixed(16);
document.getElementById("one").innerHTML = "oldSlope = " oldSlope ", newSlope = " newSlope;
document.getElementById("two").innerHTML = "oldSlopeFixed = " oldSlopeFixed ", newSlopeFixed = " newSlopeFixed;
if ((newSlope) >= (oldSlope))
{
document.getElementById("three").innerHTML = "newSlope >= oldSlope (correct)";
}
if (!((newSlopeFixed) >= (oldSlopeFixed)))
{
document.getElementById("four").innerHTML = "newSlopeFixed < oldSlopeFixed (incorrect ... what???)";
}
<!DOCTYPE html>
<html>
<body>
<p id="one"></p>
<p id="two"></p>
<p id="three"></p>
<p id="four"></p>
</body>
</html>
CodePudding user response:
toFixed
produces a string. You aren't comparing numbers, you're comparing strings, which are compared alphabetically.
console.log(-0.001 <= -0.002) // false
console.log('-0.001' <= '-0.002') // true
Don't use toFixed
before calculations. It is used when all calculations are done and you're trying to format a number for human consumption.