Home > Enterprise >  Finding the absolute value of a Byte variable in Kotlin or Java?
Finding the absolute value of a Byte variable in Kotlin or Java?

Time:06-03

Why isn't there any function in the standard library of Kotlin/Java for taking the absolute value of a Byte/byte variable? I'm I missing something?

Math.abs() is only defined for int, long, double and float.

For context: in the audio world you can run easily into byte arrays representing the amplitude. I'm interested in calculating the average of the absolute values of a byte array. For e.g see this listener related to Visualizer in Android.

I know I can cast it to an integer and take the absolute value of that, but I would still be interested why is this not predefined.

CodePudding user response:

The operations in java.lang.Math are in line with all other arithmetic operations in Java. Integer operations always work in either, 64 bit long or 32 bit int.

As stated in JLS, §4.2.2. Integer Operations

If an integer operator other than a shift operator has at least one operand of type long, then the operation is carried out using 64-bit precision, and the result of the numerical operator is of type long. If the other operand is not long, it is first widened (§5.1.5) to type long by numeric promotion (§5.6).

Otherwise, the operation is carried out using 32-bit precision, and the result of the numerical operator is of type int. If either operand is not an int, it is first widened to type int by numeric promotion.

In other words, not even the following, equivalent to abs, would compile:

byte a = 42, absA = a < 0? -a: a;

as the numeric operation -a will promote a to int before negating.

It’s important that a cast of the result to byte would not be a lossless operation here. The byte datatype has a value range from -128 to 127, so if the value is -128, its absolute value 128 is outside the byte value range and a cast to byte would cause an overflow to -127.

Therefore, to have a correct and efficient calculation, you should do as always in Java when it comes to byte, short, or char calculations, calculate everything using int and only cast the final result back to your data type. When you want to calculate the average, you have to calculate the sum using int anyway (or even long if you have more than 16777215 array elements).

byte[] array // e.g. test case: = { 1, -1, -128, 127 };

int sum = 0;
for(byte b: array) sum  = Math.abs(b);
int average = sum/array.length;
// if you really need a byte result
byte byteAverage = average == 128? 127: (byte)average;

I don’t know about Kotlin, but in Java, the automatic promotion to int also works if the operand is of type Byte, so you don’t need to “cast it to an integer” to call Math.abs(int). You only have to deal with the fact that the result will be an int, as with all arithmetic operations on byte, short, char, or their wrapper types.

CodePudding user response:

In java byte is signed between -128 and 127, corresponding as (unsigned) int: 0xFF & b between 128 .. 255, and 0 .. 127.

Math.abs is irrelevant here as probably unsigned byte values are assumed.

int[] bytesToInt(byte[] bs) {
    int[] is = new int[bs.length];
    Arrays.fill(is, i -> bs[i] & 0xFF);
    return is;
}

byte byteAbs(byte b) {
    return b >= 0? b : b == -128? 127 : -b;
}

byteAbs - given for completeness - reduces the range to 7 bits, and has the artefact that -128 maps to 127, as there is no 128.

  • Related