How to get both logits and probabilities from a custom neural network model-CodePudding

The following source code could get both probabilities and logits from an imagenet pretrained model in Tensorflow

import tensorflow as tf
import tensorflow.contrib.slim as slim
import tensorflow.contrib.slim.nets as nets
import PIL
import numpy as np

import warnings
warnings.filterwarnings('ignore')

#Sets the threshold for what messages will be logged
tf.logging.set_verbosity(tf.logging.ERROR)
#Starts the Interactive Session
sess=tf.InteractiveSession()

#Get logits and probs from the model
def inception(image, reuse):
    preprocessed = tf.multiply(tf.subtract(tf.expand_dims(image, 0), 0.5), 2.0)
    arg_scope = nets.inception.inception_v3_arg_scope(weight_decay=0.0)
    with slim.arg_scope(arg_scope):
        logits, _ = nets.inception.inception_v3(preprocessed, 1001, is_training=False, reuse=reuse)
        logits = logits[:,1:]
        probs = tf.nn.softmax(logits)
    return logits, probs

#Returns logits and probabilities from the network 
logits, probs = inception(image, reuse=False)

Now, suppose I have the following model: a RESNET-50 fine tuned in another dataset that I will simply load its model definition and weights:

json_file = open('/path/resnet-model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()

model = model_from_json(loaded_model_json)
#load weights into new model
model.load_weights("/path/weights/resnet-weights.h5")

How to make a similar function to take the probs and logits from such an existing pre-trained model?

P.s1: the model, its weights, and an example image input can be found HERE

P.s2: some existing solutions to questions like HERE only explain how to get one or another. I need a function like the first one above that will give me both the logits and probs from an existing trained model loaded from a file.

CodePudding user response：

IIUC, you should be able to do this directly the same way:

import tensorflow as tf

json_file = open('/content/resnet-model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()

loaded_model_json = loaded_model_json.replace('"activation":"softmax"', '"activation":"linear"')
model = tf.keras.models.model_from_json(loaded_model_json)

image = tf.keras.preprocessing.image.load_img('/content/sample-image.jpeg')
image = tf.constant([tf.keras.preprocessing.image.img_to_array(image)])
logits = model(image)
probs = tf.nn.softmax(logits)

You could also define a new model with the reverse of a softmax function:

def inv_softmax(x, C):
   return tf.math.log(x)   C

outputs = tf.keras.layers.Lambda(lambda x : inv_softmax(x, tf.math.log(10.)),name='inv_softmax')(model.output)
new_model = tf.keras.Model(model.input, outputs) 
logits = new_model(image)
probs = tf.nn.softmax(logits)

Or just drop the last layer and define a new one with a linear activation function.