I used FER2013 dataset from kaggle and trained a CNN model. Saved the model as TFLite. Made a Kotlin app using it. Now I am not able to get proper output. Sample output of the model : [0. 0. 0. 1. 0. 0. 0.]
for happy
Please check the code for MainActivity.kt. I am pure noobie. Thank you so much for bearing with me.
package com.example.mooddetector
import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.os.Bundle
import android.util.Log
import android.widget.Button
import android.widget.ImageView
import android.widget.TextView
import android.widget.Toast
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AppCompatActivity
import androidx.core.content.ContextCompat
import com.example.mooddetector.databinding.ActivityMainBinding
import com.example.mooddetector.ml.MoodDetector
import org.tensorflow.lite.DataType
import org.tensorflow.lite.support.image.ColorSpaceType
import org.tensorflow.lite.support.image.TensorImage
import org.tensorflow.lite.support.image.ops.TransformToGrayscaleOp
import org.tensorflow.lite.support.tensorbuffer.TensorBuffer
class MainActivity : AppCompatActivity() {
private lateinit var binding: ActivityMainBinding
private lateinit var imageView: ImageView
private lateinit var button: Button
private lateinit var tvOutput: TextView
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = ActivityMainBinding.inflate(layoutInflater)
val view = binding.root
setContentView(view)
imageView = binding.imageView
button = binding.btnCaptureImage
tvOutput = binding.tvOutput
val buttonLoad = binding.btnLoadImage
button.setOnClickListener {
if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA)
== PackageManager.PERMISSION_GRANTED
) {
takePicturePreview.launch(null)
} else {
requestPermission.launch(android.Manifest.permission.CAMERA)
}
}
}
private val requestPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
if (granted) {
takePicturePreview.launch(null)
} else {
Toast.makeText(this, "Permission Denied.", Toast.LENGTH_SHORT).show()
}
}
private val takePicturePreview = registerForActivityResult(ActivityResultContracts.TakePicturePreview()){bitmap->
if(bitmap!=null){
imageView.setImageBitmap(bitmap)
outputGenerator(bitmap)
}
}
private fun outputGenerator(bitmap: Bitmap){
val model = MoodDetector.newInstance(this)
val newBitmap = Bitmap.createScaledBitmap(bitmap, 48, 48, true)
val tfimage = TensorImage(DataType.FLOAT32)
tfimage.load(newBitmap)
val tfimagegrayscale = TransformToGrayscaleOp().apply(tfimage)
val tensorbuffr=tfimagegrayscale.tensorBuffer
val tensorimg = TensorImage(DataType.FLOAT32)
tensorimg.load(tensorbuffr,ColorSpaceType.GRAYSCALE)
val byteBuffer = tensorimg.buffer
// Creates inputs for reference.
val inputFeature0 = TensorBuffer.createFixedSize(intArrayOf(1, 48, 48, 1), DataType.FLOAT32)
inputFeature0.loadBuffer(byteBuffer)
// Runs model inference and gets result.
val outputs = model.process(inputFeature0)
val outputFeature0 = outputs.outputFeature0AsTensorBuffer
tvOutput.text = outputFeature0.toString()
Log.d("TAG", outputs.toString())
Log.d("TAG", outputFeature0.toString())
// val data1 = outputFeature0.floatArray
// Log.d("TAG2", outputFeature0.dataType.toString())
// Log.d("TAG2", data1[0].toString())
// val probabilityBuffer = TensorBuffer.createFixedSize(intArrayOf(1, 1001), DataType.UINT8)
// Releases model resources if no longer used.
model.close()
}
}
The output of the last 2 log files is: com.example.mooddetector.ml.MoodDetector$Outputs@a04fe1 org.tensorflow.lite.support.tensorbuffer.TensorBufferFloat@ca3b548
CodePudding user response:
The docs for TensorBufferFloat list 2 methods that might be useful to you
float[] getFloatArray()
Returns a float array of the values stored in this buffer.
float getFloatValue(int absIndex)
Returns a float value at a given index.
The docs are for Java, which means that in Kotlin all getters (and setters) just become normal field/property access, i.e. .getFloatArray()
becomes just .floatArray
So if you get the whole array with .floatArray
you can just join the values together with some separator to get a string representation.
val output = outputFeature0.floatArray.joinToString(", ", "[", "]")
Log.d("TAG", output)
If you want to control the formatting use the DecimalFormat
val pattern = "#.0#" // rounds to 2 decimal places if needed
val locale = Locale.ENGLISH
val formatter = DecimalFormat(pattern, DecimalFormatSymbols(locale))
formatter.roundingMode = RoundingMode.HALF_EVEN // this is the default rounding mode anyway
val output = outputFeature0.floatArray.joinToString(", ", "[", "]") { value ->
formatter.format(value)
}
Log.d("TAG", output)
If you need to do this formatting in different places you can move the logic into an extension method
fun FloatArray.joinToFormattedString(): String {
val pattern = "#.0#" // rounds to 2 decimal places if needed
val locale = Locale.ENGLISH
val formatter = DecimalFormat(pattern, DecimalFormatSymbols(locale))
formatter.roundingMode = RoundingMode.HALF_EVEN // this is the default rounding mode anyway
return this.joinToString(
separator = ", ",
prefix = "[",
postfix = "]",
) { value ->
formatter.format(value)
}
}
Then you can simply call
Log.d("TAG", outputFeature0.floatArray.joinToFormattedString())