How to get the same accuracy value as model.fit-CodePudding

In the following keras model, I added "my_accuracy" and "accuracy" into the model.compile, and then run model.fit:

import tensorflow as tf
import numpy as np

xx = np.random.rand(28,28,1).astype(np.float32)

yy = np.random.rand(28,28,1).astype(np.float32)

zz = np.random.rand(28,28,1).astype(np.float32)

x = tf.keras.layers.Input(shape=(28, 28, 1))
y = tf.keras.layers.Dense(1)(x)

model = tf.keras.Model(x, y)
model.summary()

def my_accuracy(y_true, y_pred):
    return tf.keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))

model.compile(
    optimizer = tf.keras.optimizers.Adam(),
    loss = tf.keras.losses.MeanSquaredError(),
    metrics = [
        my_accuracy,
        "accuracy",
    ]
)

history = model.fit(xx, yy, epochs = 2, verbose = 1)

Here is the output:

Model: "model_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_2 (InputLayer)        [(None, 28, 28, 1)]       0         
                                                                 
 dense_3 (Dense)             (None, 28, 28, 1)         2         
                                                                 
=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________
Epoch 1/2
WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

1/1 [==============================] - 0s 354ms/step - loss: 0.1529 - my_accuracy: 0.4566 - accuracy: 0.0000e 00
Epoch 2/2
1/1 [==============================] - 0s 4ms/step - loss: 0.1527 - my_accuracy: 0.4554 - accuracy: 0.0000e 00

And then, call the my_accuracy directly:

my_pred = model.predict(yy, verbose=0)
my_acc = my_accuracy(yy, my_pred).numpy()

print("my_acc: {}", my_acc)

And the result:

my_acc: {} [[[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 ...

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]]

For "my_accuracy":

When calling my_accuracy directly, the result is an array, but the value of model.fit is just a number 0.4566.

How to get the same result as model.fit (0.4566) when calling the method directly?

1/1 [==============================] - 0s 354ms/step - loss: 0.1529 - my_accuracy: 0.4566 - accuracy: 0.0000e 00

For "accuracy":

What is the real method of accuracy? Why is the value equal to 0.0000e 00?

For the warning:

Why this warning says shape (None, 28, 1, 1)? Where is the shape (None, 28, 1, 1)?

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

CodePudding user response：

Lot of your problems stem from a wrong input_shape. You have 28,28,1 sized data. But your model's input shape is (None, 28, 28, 1) i.e. TensorFlow automatically adds a batch dimension in the front. So let's change your model as follows.

x = tf.keras.layers.Input(shape=(28, 1))
y = tf.keras.layers.Dense(1)(x)

model = tf.keras.Model(x, y)

This fixes the model.

What is the real method of accuracy? Why is the value equal to 0.0000e 00?

You have float values in your labels (i.e. yy). Accuracy is computed with labels being 0 or 1. None of the predictions going to exactly match when they are floating values, thus you get an accuracy of 0. Replace yy with something like np.random.choice([0, 1], size=(28,28,1)) and you'll get non-zero accuracy, matching the value computed by my_accuracy.

Why this warning says shape (None, 28, 1, 1)? Where is the shape (None, 28, 1, 1)?

This is basically saying the model is not getting correct input shape. It's because of the incorrect input shape of the model.