Home > Software engineering >  JAVA integer overflow intrigue
JAVA integer overflow intrigue

Time:04-09

The 1st for loop in the below code does not find the maximum correctly due to an overflow. However, the 2nd for loop does. I used godbolt.com to look at the byte codes for this program which showed that to determine which number is greater the 1st for loop uses an isub and the 2nd for loop uses an if_icmple. Makes sense. However, why is the if_icmple able to successfully do this comparison since it too at some point must do a subtraction (which I would expect to produce an overflow)?

public class Overflow {
   public static void main(String[] args) {
      int[] nums = {3,-2147483648, 5, 7, 27, 9};
       
      int curMax = nums[0];
      for (int num : nums) {
         int diff = curMax - num; 
         if (diff < 0) {
            curMax = num;
         }
      }
      System.out.println("1) max is "   curMax);
       
      curMax = nums[0];
      for (int num : nums) {
         if (num > curMax) {
            curMax = num;
         }
      }  
      System.out.println("2) max is "   curMax);        
   }   
}

The output is

  1. max is -2147483648
  2. max is 27

CodePudding user response:

Let's say that comparison is implemented using subtraction. Contrary to various other opinions here, I'd say that is highly likely. Eg cmp on x86 is just a subtraction that does not update its destination register, only the flags. Various other (but maybe not all) processors that have a flags register also work that way. In the rest of this answer I'll use x86 as a representative processor for examples.

However, there is an incorrect assumption made implicitly by your code: a comparison is not equivalent to a subtraction followed by checking the sign, it is equivalent to a subtraction followed by checking some combination of the Zero, Sign, and Overflow flags. For example, if you implement if (num > curMax) using some cmp followed by jle (to skip the body of the if when the condition is false), then jle does this:

Jump short if less or equal (ZF=1 or SF≠ OF).

Expressing the condition SF≠ OF directly in Java is not so easy. But the JVM itself has no such problem, it can use a comparison (or, equivalently, subtraction) followed by exactly the right kind of conditional jump.

There are some less-fortunate processors that do not have such as full set of conditional jumps as x86 has, but even in that case, the JVM has a lot more options than you do.

The option that the JVM does not have though, is implementing comparison incorrectly.

CodePudding user response:

However, why is the if_icmple able to successfully do this comparison since it too at some point must do a subtraction (which I would expect to produce an overflow)?

It doesn't need to do a subtraction. It just needs to do a comparison. Comparisons don't have to involve subtraction, certainly not in the CPU.

Of course, we can also take alternate steps to deal with overflow. Here is one simple approach:

int cmp(int a, int b) {
  boolean aNeg = (a >>> 31) != 0;
  boolean bNeg = (b >>> 31) != 0;
  if (aNeg != bNeg) {
    return aNeg ? -1 : 1;
  }
  int diff = a - b; // subtracting two numbers with the same sign can't overflow
  return (diff == 0) ? 0 : (diff < 0) ? -1 : 1; 
}

All Java has to do is compile if_icmple to something like this, or whatever other instruction is appropriate on the target CPU. Using bytecode means Java can leave that up to the runtime to get right for the target CPU, whatever's fastest in such an environment -- using the overflow bit, doing something like this, whatever.

  • Related