Two identical double-precision decimal subtraction is not equal to zero-CodePudding

Home > front end > Two identical double-precision decimal subtraction is not equal to zero

Two identical double-precision decimal subtraction is not equal to zero

Time：09-16

Two CDBL type of 7.6 is presupposed, equal to

7.6-7.6=2.66453525910038 e-15

How to deal with this?

CodePudding user response:

Do round round function

CodePudding user response:

Make sure to keep a few decimal places

CodePudding user response:

Is unlikely to appear this problem, you use two variables, it will be a reduction of try