I have a certain for loop in my code (number of iterations 100). Somehow the code get stuck at iteration number 34/100. I am pretty sure that this issue is linked to these two functions that I am using in the loop,

def sigmoid(self, a: np.float128):
        """
        inputs:
            v: float

        returns:
            the logistic sigmoid evaluated at a
        """
        return 1 / (1   scipy.special.expit(-a))
        # return 1 / (1   np.exp(-a))

def loss(self, w, X, y):
        """
        inputs: w: an array of the current weights, of shape (d,)
                X: an array of n datapoints, of shape (n, d)

        outputs: the loss. This is exactly the negative log likelihood
        """
        E = 10 ** (-8)
        y_pred = self.sigmoid(np.dot(X, w))
        cost = -np.sum(y * np.log(y_pred   E)   (1 - y) * np.log(1 - y_pred   E))
        return cost

The code gives a warning at the beginning:

<ipython-input-65-26bf85bda945>:72: RuntimeWarning:

overflow encountered in exp

So I got rid of this warning by using scipy.special.expit(x) instead of np.exp(x). But that doesn't seem to solve the loop problem. Any ideas?

CodePudding user response：

Don't replace exp with expit; replace sigmoid with expit. scipy.special.expit is the logistic sigmoid function.