I'm using MSVC 2019 v16.11.12.

When I tried compiling my code with /fp:fast instead of /fp:precise, my tests started failing.

The simplest case is:

BOOST_AUTO_TEST_CASE(test_division) {
    float v = 3000.f;
    BOOST_TEST((v / 1000.f) == 3.f);
}

Failing with result:

error: in "test_division": check (v / 1000.f) == 3.f has failed [3.00000024 != 3]

I understand that /fp:fast can have worse floating-point precision in some cases; but, here, it seems excessive...

How come it can't accurately divide 3000 by 1000?

I understand that 0.1 0.2 is not 0.3, but here all the numbers are representable and the division returns exactly 3 with fp:precise.

Is it possible that I have some other flag flipped which decreased the floating-point precision even more?

CodePudding user response：

One of the optimizations which is enabled by gcc's -ffast-math option (and probably msvc's /fp:fast option) is converting a "divide by constant" into a "multiply by reciprocal", as floating point divides are quite slow -- on some machines more than 10x as expensive as a multiply, as multipliers are commonly pipelined while dividers are less commonly pipelined.

With this, the / 1000.f would get turned into a * .001 of some precision, and .001 cannot be exactly represented in floating point, so some imprecision will occur.

More precisely, the closest 32-bit FP value to .001 is 0x1.0624dep-10, while the closest 64-bit FP is 0x1.0624dd2f1a9fcp-10. If that 32-bit value is multiplied by 3000 you'll get 0x1.80000132p 1 or about 3.0000001425. If you round that to 32 bits, you'll get 0x1.800002p 1 or about 3.0000002384. Interestingly, if you use the 64-bit value and multiply by 3000, you'll get the exact 0x1.8p 1 value.