Converting a Keras model to TensorFlow lite - how to avoid unsupported operations?-CodePudding

I have MobileNetV2 based model that uses the TimeDistributed layer. I want to convert that model to a TensorFlow Lite model in order to run it on a smartphone, but there is on undefined operation.

Here is the code:

import tensorflow as tf

IMAGE_SHAPE = (224, 224, 3)

mobilenet_model = tf.keras.applications.MobileNetV2(input_shape=IMAGE_SHAPE,
                                               include_top=False,
                                               pooling='avg',
                                               weights='imagenet')

inputs = tf.keras.Input(shape=(5,)   IMAGE_SHAPE)
x = tf.keras.applications.mobilenet_v2.preprocess_input(inputs)
outputs = tf.keras.layers.TimeDistributed(mobilenet_model)(x)
model = tf.keras.Model(inputs, outputs)
model.compile()
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tfmodel = converter.convert() # fails

Here is the error message:

error: failed while converting: 'main': 
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select 
TF Select ops: Mul
Details:
    tf.Mul(tensor<?x5x224x224x3xf32>, tensor<f32>) -> (tensor<?x5x224x224x3xf32>)

The error is caused by the interaction between the input preprocessing and TimeDistributed layer. If I disable the input preprocessing, then the conversion succeeds, but obviously the network will not work properly without retraining. Also models that have the preprocessing but don't have TimeDistributed layer can be converted. Is it possible to move the preprocessing to a different place to avoid this error?

Also, adding the select ops helps to convert it successfully, but I'm not sure how to enable them at runtime. I'm using the Mediapipe framework to build an Android app. and I don't think Mediapipe supports linking to extra operations.

CodePudding user response：

Try this to conversion of model.

converter = TFLiteConverter.from_saved_model(model_dir)
converter.experimental_new_converter = False
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
converter.allow_custom_ops=True
tflite_model = converter.convert()

CodePudding user response：

Check the documentation on how to convert it and use it inside android. Maybe you will have success with that. If not come back and ping me again.

You can give it a try without:

converter.target_spec.supported_ops = [
  tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
  tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]

if your tensorflow version is 2.6.0 or 2.6.1

Check the allow list operators here

Best