Add preprocess layer to the pretrained keras model after the input layer-CodePudding

I'm buliding an ensemble model with few pretrained keras applications (ResNet50, DenseNet) and when I try to load the saved keras applications for testing I face an issue as different pre trained models require different input preprocessing and it causes to reduce the accuracy of the models.

Hence I thought of adding a pre process layer after the input layer to both pretrained models separately to perform the preprocessing from within the model and avoid any preprocessing from the ImageDataGenerator using the
tf.keras.applications.---model name---.preprocess_input.

So I'm wondering how should I do this task.

resNet = tf.keras.applications.ResNet50(
    include_top=False,
    weights="imagenet",
    input_shape=(256,256,3),
    pooling="avg",
    classes=13,
)

for layer in resNet.layers:
    layer.trainable = False

model = Sequential()

model.add(resNet)
model.add(Flatten())
model.add(Dense(22, activation='softmax',name='output'))

This is how the code looks as of now and I need to add a preprocess layer for the relevant preprocessing required for ResNet50(according to the above scenario).

Sequential Model after adding the dense layer

Please help me to solve this problem.

CodePudding user response：

Adding a preprocessing layer after the Input layer is the same as adding it before the ResNet50 model,

resnet = tf.keras.applications.ResNet50( 
    include_top=False ,
    weights='imagenet' ,
    input_shape=( 256 , 256  , 3) ,
    pooling='avg' ,
    classes=13
)
for layer in resnet.layers:
    layer.trainable = False

# Some preprocessing layer here ...
preprocessing_layer = tf.keras.layers.Normalization( mean=0 , variance=1 , input_shape=( 256 , 256 , 3 ) )

model = tf.keras.models.Sequential( [
     preprocessing_layer ,
     resnet,
     tf.keras.layers.Flatten(),
     tf.keras.layers.Dense(22, activation='softmax',name='output') 
])
model.compile( loss='mse' , optimizer='adam' )
model.summary()

The output,

_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalization (Normalizatio  (None, 256, 256, 3)      0         
 n)                                                              
                                                                 
 resnet50 (Functional)       (None, 2048)              23587712  
                                                                 
 flatten (Flatten)           (None, 2048)              0         
                                                                 
 output (Dense)              (None, 22)                45078     
                                                                 
=================================================================
Total params: 23,632,790
Trainable params: 45,078
Non-trainable params: 23,587,712
_________________________________________________________________

The inputs will now go through the preprocessing_layer first, and then to the resnet model.

CodePudding user response：

If you want to use tf.keras.applications.resnet50.preprocess_input(), try:

import tensorflow as tf

resnet = tf.keras.applications.ResNet50( 
    include_top=False ,
    weights='imagenet' ,
    input_shape=( 256 , 256  , 3) ,
    pooling='avg' ,
    classes=13
)
for layer in resnet.layers:
    layer.trainable = False

model = tf.keras.Sequential()
model.add(tf.keras.layers.Lambda(tf.keras.applications.resnet50.preprocess_input, input_shape=(256, 256, 3)))
model.add(resnet)
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(22, activation='softmax',name='output') )

model.compile( loss='mse' , optimizer='adam' )
print(model(tf.random.normal((1, 256, 256, 3))))

tf.Tensor(
[[0.12659772 0.02955576 0.13070999 0.0258545  0.0186768  0.01459627
  0.07854564 0.010685   0.01598095 0.04758708 0.05001146 0.20679766
  0.00975605 0.01047837 0.00401289 0.01095579 0.06127766 0.0313729
  0.00884041 0.04098257 0.01187507 0.05484949]], shape=(1, 22), dtype=float32)