Syth90k training data to do the training, then trained to appear in the process of model output is nan, and it is time to the 10 batches of data will appear this problem, I disturb data is also awarded in 10 nan, my bs Settings is 10, when adjusted to 1 in more than 700 of nan, I checked the input data, all is normal, and the most strange is, if I'm in training after the 9 group of data store model, and then reload, nan won't immediately appear, will appear on the training again after a period of time

CodePudding user response:

reference 1st floor ITMessager response:

problem solved? Specific problems may require specific analysis, if necessary, can learn,

is not solve yet

CodePudding user response:

refer to the second floor zzy_19960503 response:

is not solve yet

can add me q1099968024, exchange of learning

CodePudding user response:

The building Lord, you delete my comment on dry ha ah??
Because I am training code random one epoch into a NAN, want to communicate with you learn ah ~

CodePudding user response:

Solve the building Lord, I and your problem now, is in particular a appeared a batch of nan