Home > Back-end >  Need precision out to 25th decimal, Python keeps rounding down when computing
Need precision out to 25th decimal, Python keeps rounding down when computing

Time:09-09

My code is essentially: arrA=arrB-arrC*varD. However, what happens after running that is that arrA ≠ arrB-arrC*varD, and instead arrA = arrB. I think arrC*varD is getting dropped off since it's super small, like 1e-20 at some point. However, arrA is also at like 1e-14 and I really need that precision. The arrays are float64, so I'm not sure why they're getting rounded and dropped off like that in computing.

Does anyone know how I can avoid this and keep precision out to at least the 25th decimal place? I've tried np.around to the 25th place but that's not helping either.

CodePudding user response:

use the python "decimal" module

    from decimal import *
    getcontext().prec = 30 #digits of precision that you want for your decimal object (default is 28)
    x = Decimal(1.2342346235723451432)/Decimal(4.326243232) #just an example problem -- just declare each value in your expression as a "Decimal" type to hold their precision
    #x is now a Decimal type object with precision of 30

CodePudding user response:

This has got to be already answered, think of a similar situation -pi

In intro to CS we went over this but there’s much easier methods, the simplest(laziest

  • Related