Error while compiling a model using the adam optimizer in tensorflow keras-CodePudding

I'm trying to build a ResNet50 model using the Adam optimizer. This is my code:

import tensorflow as tf
  cifar = tf.keras.datasets.cifar100
  (x_train, y_train), (x_test, y_test) = cifar.load_data()
  model = tf.keras.applications.ResNet50(
      include_top=True,
      weights=None,
      input_shape=(32, 32, 3),
      classes=100,)

  loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
  model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
  model.fit(x_train, y_train, epochs=5, batch_size=64)

But when I run this it gives the following error:

tensorflow.python.framework.errors_impl.NotFoundError: Graph execution error:

I've tried changing:

optimizer="adam"

to:

optimizer=tf.keras.optimizers.Adam

But I get another error:

ValueError: Could not interpret optimizer identifier: <class 'keras.optimizers.optimizer_experimental.adam.Adam'>

I've searched online but didn't find an answer. Any help?

CodePudding user response：

The problem should not be from the optimizer. For using from_logits=True properly, it is possible to pass classifier_activation=False argument to tf.keras.applications.ResNet50 to return the logits of the "top" layer (TensorFlow documenation):

import tensorflow as tf
cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
    include_top=True,
    weights=None,
    input_shape=(32, 32, 3),
    classes=100,
    classifier_activation=None,
)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=1, batch_size=64)

Output:

782/782 [==============================] - 2259s 3s/step - loss: 4.7819 - accuracy: 0.0762

Alternatively, one can use softmax as the activation, but with from_logits=False:

import tensorflow as tf
cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
    include_top=True,
    weights=None,
    input_shape=(32, 32, 3),
    classes=100,
    classifier_activation='softmax',
)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=1, batch_size=64)

Output:

782/782 [==============================] - 2350s 3s/step - loss: 4.7012 - accuracy: 0.0830

CodePudding user response：

I use Python 3.10 and Tensorflow 2.11.0. Code is working only there is a user warning about loss function

"UserWarning: "sparse_categorical_crossentropy received from_logits=True, but the output argument was produced by a Softmax activation and thus does not represent logits. Was this intended? output, from_logits = _get_logits("

Maybe you can try "loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)"

import tensorflow as tf
cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.resnet50.ResNet50(
      include_top=True,
      weights=None,
      input_shape=(32, 32, 3),
      classes=100,)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=5, batch_size=64)