I have a multi-class classifier that I also want to query for the output of one of the intermediate layers.
inputs = Input(...)
...
fc = Dense(32, activation='relu', name='FC_1')(layer)
x = Dense(num_cats, activation='softmax', name='Softmax')(fc)
outputs = {
'predictions': x,
'fc': fc,
}
model = Model(inputs=inputs, outputs=outputs)
opt = Adam()
cat_crossentropy = SparseCategoricalCrossentropy()
loss = {
'predictions': cat_crossentropy,
}
model.compile(optimizer=opt, loss=loss, metrics={'predictions': [
SparseCategoricalAccuracy(),
SparseTopKCategoricalAccuracy(k=3),
]})
This works great while in the same process. model.fit() does what you'd expect, and model.predict() returns outputs with the values for the two keys defined.
However, model.save(output_path) raises the following exception:
ValueError: Unable to save the object {
'predictions': <keras.losses.SparseCategoricalCrossentropy object at 0x15e5c6c70>,
'dense': None,
}
(a dictionary wrapper constructed automatically on attribute assignment).
The wrapped dictionary was modified outside the wrapper (its final value was
{
'predictions': <keras.losses.SparseCategoricalCrossentropy object at 0x15e5c6c70>,
'dense': None
},
its value when a checkpoint dependency was added was None),
which breaks restoration on object creation.
If you don't need this dictionary checkpointed,
wrap it in a non-trackable object; it will be subsequently ignored.
Question
How do I exported a saved model containing named outputs?
Edit - reproducible code
import tensorflow as tf
import numpy as np
from tensorflow.data import Dataset
from tensorflow.keras import Model, Input
from tensorflow.keras.layers import Dense
from tensorflow.keras.metrics import SparseCategoricalAccuracy
from tensorflow.keras.losses import SparseCategoricalCrossentropy
def build_model():
i = Input(shape=(5,))
fc = Dense(5, activation='relu')(i)
softmax = Dense(5, activation='softmax', name='Softmax')(fc)
outputs = {'predictions': softmax, 'dense': fc}
return Model(inputs=i, outputs=outputs)
model = build_model()
opt = tf.keras.optimizers.Adam(learning_rate=0.003)
model.compile(optimizer=opt, metrics={'predictions': [SparseCategoricalAccuracy()]}, loss={
'predictions': SparseCategoricalCrossentropy(from_logits=False)
})
_ds = Dataset.from_tensor_slices((np.random.rand(10, 5), np.random.randint(0, 5, 10)))
model.fit(_ds.batch(32), epochs=3, verbose=0)
print(model.predict([[0.1] * 5]))
# >>> {
# 'predictions': array([[
# 0.20790772,
# 0.18233214,
# 0.18703228,
# 0.20266992,
# 0.22005796
# ]], dtype=float32),
# 'dense': array([[
# 0.1394624,
# 0.16406271,
# 0.,
# 0.,
# 0.
# ]], dtype=float32)}
model.save('my_model')
# >>> Traceback (most recent call last): ...
CodePudding user response：
This worked for me, let me know if it works for you. This helped me figure it out.
import tensorflow as tf
import numpy as np
from tensorflow.data import Dataset
from tensorflow.keras import Model, Input
from tensorflow.keras.layers import Dense
from tensorflow.keras.metrics import SparseCategoricalAccuracy
from tensorflow.keras.losses import SparseCategoricalCrossentropy
def build_model():
i = Input(shape=(5,))
fc = Dense(5, activation='relu')(i)
softmax = Dense(5, activation='softmax', name='Softmax')(fc)
# outputs = {'predictions': softmax, 'dense': fc}
left_output = softmax
right_output = fc
return Model(inputs=i, outputs=[left_output, right_output])
model = build_model()
opt = tf.keras.optimizers.Adam(learning_rate=0.003)
# model.compile(optimizer=opt, metrics={'predictions': [SparseCategoricalAccuracy()]}, loss={
# 'predictions': SparseCategoricalCrossentropy(from_logits=False)
# })
model.compile(optimizer=opt, metrics="SparseCategoricalAccuracy", loss=[SparseCategoricalCrossentropy(from_logits=False)])
_ds = Dataset.from_tensor_slices((np.random.rand(10, 5), np.random.randint(0, 5, 10)))
model.fit(_ds.batch(32), epochs=3, verbose=1)
print(type(model.predict([[0.1] * 5])))
print(model.predict([[0.1] * 5]))
# >>> {
# 'predictions': array([[
# 0.20790772,
# 0.18233214,
# 0.18703228,
# 0.20266992,
# 0.22005796
# ]], dtype=float32),
# 'dense': array([[
# 0.1394624,
# 0.16406271,
# 0.,
# 0.,
# 0.
# ]], dtype=float32)}
model.save('my_model')
# >>> Traceback (most recent call last): ...
Output
Epoch 1/3
1/1 [==============================] - 0s 207ms/step - loss: 1.7489 - >Softmax_loss: 1.7489 - Softmax_sparse_categorical_accuracy: 0.3000 - >dense_4_sparse_categorical_accuracy: 0.4000
Epoch 2/3
1/1 [==============================] - 0s 1ms/step - loss: 1.7379 - >Softmax_loss: 1.7379 - Softmax_sparse_categorical_accuracy: 0.3000 - >dense_4_sparse_categorical_accuracy: 0.4000
Epoch 3/3
1/1 [==============================] - 0s 920us/step - loss: 1.7271 - >Softmax_loss: 1.7271 - Softmax_sparse_categorical_accuracy: 0.3000 - >dense_4_sparse_categorical_accuracy: 0.4000
WARNING:tensorflow:5 out of the last 7 calls to Model.make_predict_function..predict_function at >0x7f229d14df70> triggered tf.function retracing. Tracing is expensive >and the excessive number of tracings could be due to (1) creating >@tf.function repeatedly in a loop, (2) passing tensors with different >shapes, (3) passing Python objects instead of tensors. For (1), >please define your @tf.function outside of the loop. For (2), >@tf.function has experimental_relax_shapes=True option that relaxes >argument shapes that can avoid unnecessary retracing. For (3), please >refer to >https://www.tensorflow.org/guide/function#controlling_retracing and >https://www.tensorflow.org/api_docs/python/tf/function for more >details.
<class 'list'>
[array([[0.1952633 , 0.19603369, 0.18926372, 0.2137877 , >0.20565161]],
dtype=float32), array([[0.12314494, 0. , 0.13512844, 0. >, 0. ]],
dtype=float32)]
INFO:tensorflow:Assets written to: my_model/assets
CodePudding user response：
Found a solution and a gotcha.
Solution
The solution is simple: instead of omitting keys in the losses dict used in model.compile for outputs that don't need a loss (intermediate layers), add the key pointing at a null value.
model.compile(
optimizer=opt,
metrics={'predictions': [
SparseCategoricalAccuracy(),
SparseTopKCategoricalAccuracy(k=3)
]},
loss={
'predictions': SparseCategoricalCrossentropy(from_logits=False),
'fc': None,
# ^^^^^^^^^^^
# The solution
})
Now the output of model.predict() is a dict with the same keys as in model.outputs. Loading the model with tf.keras.models.load_model() just works.
Gotcha
In Tensorflow Serving, when you perform inference, the output will be a dict whose values are the Layer.name values in the layers referenced in model.outputs.
# Layer names: ("FC_1", "Softmax")
fc = Dense(32, activation='relu', name='FC_1')(layer)
x = Dense(num_cats, activation='softmax', name='Softmax')(fc)
# Keys in model.outputs: ("predictions", "fc")
outputs = {
'predictions': x,
'fc': fc,
}
model = Model(inputs=inputs, outputs=outputs)
When you evaluate the model via Tensorflow Serving, the output will be of the form:
{
"outputs": {
"Softmax": [ [ 0.1, ... ] ],
"FC_1": [ [...] ]
}
}
You can also see what the model's output layers are in Tensorflow Serving using the /v1/models/{modelId}/versions/1/metadata REST endpoint.