My code is essentially:
arrA=arrB-arrC*varD
.
However, what happens after running that is that arrA ≠ arrB-arrC*varD
, and instead arrA = arrB
. I think arrC*varD
is getting dropped off since it's super small, like 1e-20 at some point. However, arrA
is also at like 1e-14 and I really need that precision. The arrays are float64, so I'm not sure why they're getting rounded and dropped off like that in computing.
Does anyone know how I can avoid this and keep precision out to at least the 25th decimal place? I've tried np.around to the 25th place but that's not helping either.
CodePudding user response:
use the python "decimal" module
from decimal import *
getcontext().prec = 30 #digits of precision that you want for your decimal object (default is 28)
x = Decimal(1.2342346235723451432)/Decimal(4.326243232) #just an example problem -- just declare each value in your expression as a "Decimal" type to hold their precision
#x is now a Decimal type object with precision of 30
CodePudding user response:
This has got to be already answered, think of a similar situation -pi
In intro to CS we went over this but there’s much easier methods, the simplest(laziest