Home > Software design >  Understanding what FE_TONEAREST does
Understanding what FE_TONEAREST does

Time:03-04

The gcc documentation here, explains what FE_TONEAREST does:

This is the default mode. It should be used unless there is a specific need for one of the others. In this mode results are rounded to the nearest representable value. If the result is midway between two representable values, the even representable is chosen. Even here means the lowest-order bit is zero.

Now, consider the below example:

printf("%.2f\n", 1.235); // nearest are 1.23 and 1.24, 1.24 should be chosen
printf("%.2f\n", 1.225); // nearest are 1.22 and 1.23, 1.22 should be chosen

Based on the documentation, I expect to get 1.24 and 1.22. Instead, I get 1.24 and 1.23.

What am I missing?

CodePudding user response:

1.235 and 1.225 are not numbers in your program.

Your C implementation almost certainly uses the IEEE-754 “double precision” format for double, which is also called binary64. If it performs correctly rounded conversions to the nearest representable values, then the source text 1.235 is converted to the double 1.2350000000000000976996261670137755572795867919921875. When printf converts this to a decimal numeral with two decimal places, there is no tie; it is above 1.235, so the result is the numeral “1.24”.

The source text 1.225 is converted to the double 1.225000000000000088817841970012523233890533447265625. When printf converts this to a decimal numeral with two decimal places, there is no tie; it is above 1.225, so the result is the numeral “1.23”.

CodePudding user response:

If you print 17 digits after the decimal, you'll get this:

1.23500000000000010
1.22500000000000009

So both are slightly more than the "expected" decimal representation, and so both round up.

  • Related