LSTM to predict time sequence for multiple regions-CodePudding

I am using an LSTM to predict a sequence of 31 days for 30 regions. So output size should be 31 x 30. I have the following train_shapes:

X_train.shape
(500, 169, 30)
y_train.shape
(500, 31, 30)

The model is:

metric = metrics.MAPE
opt = optimizers.RMSprop(learning_rate=0.005)

model = models.Sequential()
model.add(layers.LSTM(30, return_sequences=True, activation='tanh'))
model.add(layers.Dense(5, activation='relu'))
model.add(layers.Dense(31, activation='linear'))

model.compile(loss='mse', 
              optimizer=opt, 
              metrics=[metric])

model.fit(X_train, y_train,
        validation_split=0.3,
        epochs=5, 
        batch_size=32,
        callbacks=[es], verbose=0)

However, when I fit, it returns:

    ---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [135], in <cell line: 1>()
----> 1 model.fit(X_train, y_train,
      2             validation_split=0.3,
      3             epochs=5, 
      4             batch_size=32,
      5             callbacks=[es], verbose=0)

File ~/.pyenv/versions/3.8.12/envs/event-forecast-rio/lib/python3.8/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File ~/.pyenv/versions/3.8.12/envs/event-forecast-rio/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py:1147, in func_graph_from_py_func.<locals>.autograph_handler(*args, **kwargs)
   1145 except Exception as e:  # pylint:disable=broad-except
   1146   if hasattr(e, "ag_error_metadata"):
-> 1147     raise e.ag_error_metadata.to_exception(e)
   1148   else:
   1149     raise

    ValueError: Dimensions must be equal, but are 169 and 31 for '{{node mean_squared_error/SquaredDifference}} = SquaredDifference[T=DT_FLOAT](sequential_24/dense_3/BiasAdd, IteratorGetNext:1)' with input shapes: [?,169,31], [?,31,30].

How could I modify the model in a way to return 31x30 matrix?

CodePudding user response：

You can try using tf.keras.layers.RepeatVector to get the right number of timesteps. Check the docs for more information. Here is a working example:

import tensorflow as tf

# Create random data
X_train = tf.random.normal((500, 169, 30))
y_train = tf.random.normal((500, 31, 30))

metric = tf.keras.metrics.MAPE
opt = tf.keras.optimizers.RMSprop(learning_rate=0.005)

model = tf.keras.Sequential()
model.add(tf.keras.layers.LSTM(30, return_sequences=False, activation='tanh'))
model.add(tf.keras.layers.RepeatVector(31))
model.add(tf.keras.layers.Dense(5, activation='relu'))
model.add(tf.keras.layers.Dense(30, activation='linear'))

model.compile(loss='mse', 
              optimizer=opt, 
              metrics=[metric])

model.fit(X_train, y_train,
        validation_split=0.3,
        epochs=5, 
        batch_size=32)

Epoch 1/5
11/11 [==============================] - 8s 247ms/step - loss: 1.0009 - mean_absolute_percentage_error: 115.8113 - val_loss: 1.0010 - val_mean_absolute_percentage_error: 106.3806
Epoch 2/5
11/11 [==============================] - 1s 135ms/step - loss: 0.9990 - mean_absolute_percentage_error: 107.6553 - val_loss: 1.0013 - val_mean_absolute_percentage_error: 121.5136
Epoch 3/5
11/11 [==============================] - 1s 128ms/step - loss: 0.9988 - mean_absolute_percentage_error: 109.4043 - val_loss: 1.0014 - val_mean_absolute_percentage_error: 117.5714
Epoch 4/5
11/11 [==============================] - 2s 154ms/step - loss: 0.9986 - mean_absolute_percentage_error: 111.2661 - val_loss: 1.0014 - val_mean_absolute_percentage_error: 114.1457
Epoch 5/5
11/11 [==============================] - 2s 180ms/step - loss: 0.9986 - mean_absolute_percentage_error: 113.6251 - val_loss: 1.0014 - val_mean_absolute_percentage_error: 116.9521
<keras.callbacks.History at 0x7f292901ecd0>