Java / Kotlin - best approach for fast pixel wise image operations-CodePudding

Overall Problem - I want to do fast image modification in Android, and my current naive solution is too slow

I'm building an Android app that will involve fast modification of streaming images. I am running it on a Samsung Galaxy S10. I'm new to Android, Java, and Kotlin, so please forgive any ignorance on my part.

I've successfully decoded the video such that each frame is read into a Bitmap.

As a simple test, I've made the following function:

 fun corruptBitmapInplace(bitmap: Bitmap){
     println("Corrupting bitmap of size ${bitmap.width} x ${bitmap.height}, heheheh")
     val start = System.currentTimeMillis()
     for (x in 0..500){
         for (y in 0..200){
             bitmap.setPixel(x, y, bitmap.getPixel(x, y) and 0xffff00)  // Remove blue channel
         }
     }
    println("Corruption took ${System.currentTimeMillis()-start}ms")
}

Which outputs

I/System.out: Corrupting bitmap of size 1280 x 720, heheheh
I/System.out: Corruption took 60ms

However it's much slower than I expected - at around 0.6us/pixel, it would take around 0.5s per image (I use 500 x 200 in this demo because when I use the full image size, the thread running the function seems to be killed before completing)

For comparison, doing the same thing in Python...:

import numpy as np
import time
image = np.random.randint(255, size=(720, 1280))
tstart = time.time()
image[:200, :500] &= 0xffff00
print(f'Elapsed: {(time.time()-tstart)*1000:.2f}ms')

... on my MacBook Air takes around 0.3ms (vs the 60ms on Galaxy 10 in Kotlin.).

So, question - what is the standard, efficient way to do such things in Android? Should I not be using native kotlin and instead something like MultiK? Or am I just doing things in an inefficient way natively?

CodePudding user response：

Nested for loops (Java/Kotlin) are done in single thread so not necessarily the most performant.

If standard bitmap, there are functions like ColorMatrixFilter but you are trying to do more with a stream of images.

Android recently depreciated their compute system Renderscript and advise moving to Vulkan. Open GL ES remains supported, see https://github.com/google/grafika and https://github.com/cats-oss/android-gpuimage for some examples.

You can write your own code with the Android NDK in C/C via JNI

OpenCV exists for Android but be aware that features between versions vary if you are porting code over from desktop.

For a machine learning based image processing model i.e. Tensorflow Lite, you can use CameraX Image Analysis if using the device camera. Other models are available such as Google's MediaPipe project.

CodePudding user response：

So it looks like it is much faster to operate on the pixels as an array. This implementation is over 100x faster:

 fun corruptBitmapInplace(bitmap: Bitmap){
     println("Corrupting bitmap of size ${bitmap.width} x ${bitmap.height}, heheheh")
     val start = System.currentTimeMillis()

     val pixels = IntArray(bitmap.width*bitmap.height)
     bitmap.getPixels(pixels, 0, bitmap.width, 0, 0, bitmap.width, bitmap.height)
     for (i in 0 until pixels.size){
         pixels[i] = pixels[i] and 0xffff00  // Remove blue channel
     }
     bitmap.setPixels(pixels, 0, bitmap.width, 0, 0, bitmap.width, bitmap.height)
     

    println("Corruption took ${System.currentTimeMillis()-start}ms")

... at around 4ms per call (doing every pixel) - as opposed to around 550ms per call when using BitMap.setPixel