Home > Net >  // (floordiv) operator gives wrong result even though fraction is a float-representable integer
// (floordiv) operator gives wrong result even though fraction is a float-representable integer

Time:10-03

def show(x):
    return "{:.50f}".format(x)

>>> show((9/50)/(1/50))
'9.00000000000000000000000000000000000000000000000000'
>>> show((9/50)//(1/50))
'8.00000000000000000000000000000000000000000000000000'

Why does this happen? How is the floordiv function implemented to yield this result?

CodePudding user response:

rounding errors in Python floor division

Please check this thread , using float as input to floor division should be done in a little tricky way

CodePudding user response:

This is the hangup:

>>> math.fmod(9/50, 1/50)
0.01999999999999999

That is, the remainder is not 0 when computed to infinite precision. Remember that things like 1/50 are represented internally as binary approximations to the decimal values. Operations like // and fmod() see the approximations.

A consequence:

>>> divmod(9/50, 1/50)
(8.0, 0.01999999999999999)

The first part of that tuple (8.0) is what // returns.

These are the exact values you're working with (every binary float can be represented exactly as a decimal float, but not always vice versa):

>>> import decimal
>>> decimal.getcontext().prec = 500
>>> a = decimal.Decimal(9 / 50)
>>> a
Decimal('0.179999999999999993338661852249060757458209991455078125')
>>> b = decimal.Decimal(1 / 50)
>>> b
Decimal('0.0200000000000000004163336342344337026588618755340576171875')

Then you can see that their quotient is very close to, but strictly less than, 9:

>>> a / b
Decimal('8.9999999999999994795829572069578825097785927606294264409785130112132181330918190728686667562468053202101562430796913250703133371819349483407279064891778548444542555094951793065257796799431977448531572173096496447578542537338521354220252562619824630430214685714904931305685046145118086722731059777831001898809747580797140817173965632373555310050843739628587610364851425663859425151431557846221951824825835845421021824148219867951326908196293925437792528353996177649543157087221511093517505990964829850')

That's why // returns 8. The remainder then is:

>>> a - 8*b
Decimal('0.0199999999999999900079927783735911361873149871826171875000')
>>> float(_)
0.01999999999999999

If you can't live with shallow surprises like this when working with conceptual decimal numbers, use the decimal module instead. The nature of binary floating-point isn't going to change ;-)

  • Related