Custom Keras loss to minimize count of elements above a given threshold-CodePudding

I am trying to create a custom loss function for a regression problem that would minimize the number of elements that falls above a certain threshold. my code for this is:

import tensorflow as tf
epsilon = 0.000001

def custom_loss(actual, predicted):     # loss
    actual = actual * 12
    predicted = predicted * 12

    # outputs a value between 1 and 20
    vector = tf.sqrt(2 * (tf.square(predicted - actual   epsilon)) / (predicted   actual   epsilon))

    # Count number of elements above threshold value of 5
    fail_count = tf.cast(tf.size(vector[vector>5]), tf.float32)

    return fail_count

I however, run into the following error:

ValueError: No gradients provided for any variable: ...

How do I solve this problem?

CodePudding user response：

I don't think you can use this loss function, because the loss does not vary smoothly as the model parameters vary - it will jump from one value to another different value as parameters pass a theshold point. So tensorflow can't calculate gradients, and so can't train the model.

It's the same reason that 'number of images incorrectly classified' isn't used as a loss function, and categorical cross-entropy, which does vary smoothly as parameters change, is used instead.

You may need to find a smoothly varying function that approximates what you want.

[Added after your response below...]

This might do it. It becomes closer to your function as temperature is reduced. But it may not have good training dynamics, and there could be better solutions out there. One approach might be to start training with relatively large temperature, and reduce it as training progresses.

temperature = 1.0
fail_count=tf.reduce_sum(tf.math.sigmoid((vector-5.)/temperature))