Why does my LSTM model predict wrong values although the loss is decreasing?-CodePudding

The

The model was compiled with model.compile(loss="mse", optimizer="adam") so my loss function is simply the mean squared error and as an optimizer I'm using Adam.

Current results

Training of the model works fine and I can see that the loss and validation loss decreases after some epochs. The actual problem occurs when I want to predict some data y_verify from some data x_verify. I do this after the training is finished to determine how well the model is trained. In the following example I simply used the data I used for training to determine how well the model is trained (I know about overfitting and that verifying with the training set is not the right way of doing it, but that is not the problem I want to demonstrate right not).

In the following graph you can see the y data I provided to the model in blue. The orange line is the result of calling model.predict(x_verify) where x_verify is of the same shape as x_train.

I also calculated the mean absolute percentage error (MAPE) of my prediction and the actual data and it came out to be around 4% which is not bad, because I only trained for 40 epochs. But this result still is not helpful at all because as you can see in the graph above the curves do not match at all.

Question:

What is going on here?

Am I using an incorrect loss function?

Why does it seem like the model tries to predict a single value for all samples rather than predicting a different value for all samples like it's supposed to be?

Ideally the prediction should be the y data which I provided so the curves should look the same (more or less).

Do you have any ideas?

Thanks! :)

CodePudding user response：

From the notebook it seems you are not scaling your data. You should normalize or standardize your data before training your model.

30 Epochs, with normalization

Worse off.

2000(!) epochs, no normalization

Okay, now the predictions are at least in the ballpark

2000 epochs, with normalization

And now the model seems to be starting to figure things out, like we'd hope it should. Given, this is training on the 11 samples that were cobbled together in the notebook, so it's naturally going to overfit. We're just happy to see it learn something.

2000 epochs, normalization, different loss

Never be afraid to try out different losses, as

Caution! Don't compare loss values between different losses. They're not on the same scale.

2000 epochs, normalization, larger learning rate

Okay, so it's taking a long time to learn. Can I nudge it along a little faster? Sure, up the learning rate of the optimizer, and it'll get you to where you're going faster. Here, we up it by a factor of 5.

model.compile(loss="mse", optimizer=tf.keras.optimizers.Adam(learning_rate=0.005))

You could even employ a learning rate scheduler that starts big and slowly diminishes it over the course of epochs.

def scheduler(epoch, lr): if epoch < 400: return lr else: return lr * tf.math.exp(-0.01) lrs = tf.keras.callbacks.LearningRateScheduler(scheduler) history = model.fit(x=x_train, y=y_train, epochs=1000, callbacks=[lrs])

Hope this all helps!

CodePudding user response：

i think it would be helpful if you change loss into huber loss and even change optimizer into sgd and then first try out to define the best learning rate based on a callback (learning rate schedule) cause of small dataset and even normalize or standardize data before training model.