I am fine tuning the distilbert pretrained model for sentiment analysis (multilabel with 6 labels) using Huggingface emotion dataset. I am new to this, but 1 epoch, 250 steps takes around 2 hours to train on Google Colab notebook, is this normal? The train dataset has 16.000 twitter text data which of course affects the performance but isn't this too long? What is the reason behind this?

Also after 3 epochs, the accuracy started to drop. What could be the reason for this?

CodePudding user response：

Are you using a GPU? If not, it's normal that it would take this much time. Also, I wouldn't be bothered by the accuracy if you're not able to run it for more epochs.