CodePudding user response:
Gradient explosion appeared in the process of training with some subtle signals, such as:Model cannot be obtained from the training data update (such as low loss),
Model is not stable, lead to losses in the process of updating significant change,
In the process of training model loss into a NaN,
If you find these questions, then you need to take a closer look at whether gradient explosion problem,
Here are some a little bit clear signal, help to confirm whether gradient explosion problem,
Training model in the process of gradient get bigger,
NaN value model weights in the training process,
In the process of training, each node layer and the error of the gradient value that persists for more than 1.0,