How to solve ValueError in LSTM input?-CodePudding

I have the training and testing numpy arrays in the following shapes

TrainX = (1234, 50, 50) Type: <class 'numpy.ndarray'> # 1234 arrays of 50 by 50 floats
TrainY = (1234, 2) Type: <class 'numpy.ndarray'>
# TrainY was one column of binary class 0 or 1. Converted it through to_categorical()

TestX = (123, 50, 50) Type: <class 'numpy.ndarray'>
TestY = (123, 2) Type: <class 'numpy.ndarray'>

I use the following code for the LSTM,

from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import LSTM, Dense, Dropout

model = Sequential()
model.add(LSTM(50, input_shape=(TrainX.shape[1], TrainX.shape[2])))
model.add(Dense(50))
model.add(Dropout(0.3))
model.add(Dense(2, activation="softmax"))

model.compile("adam", "categorical_crossentropy", metrics=["accuracy"])

model.fit(
    TrainX,
    TrainY,
    batch_size=24,
    epochs=48,
    validation_data=[np.asarray(TestX).all(), np.asarray(TestY).all()],
    class_weight=classweights,#calculated class weights
    verbose=2,
)

First I used TrainX and TrainY but got an error ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Based on this Keras LSTM input ValueError: Shapes are incompatible, I used np.asarray(TrainX). But got the same error.
So I added np.asarray(TrainX).all() one time and np.asarray(TrainX).any() another time. But got a different value error: ValueError: Failed to find data adapter that can handle input: <class 'numpy.bool_'>, <class 'numpy.bool_'>
Finally, I tried pandas.DataFrame(TrainX) for input. But it showed the following error, ValueError: Must pass 2-d input. shape=(1234, 50, 50)

trace back of the errors 1 and 2:

Traceback (most recent call last):

  File "C:\Users\...\main.py", line 73, in <module>
    lmodel.lstm(TrainX, TestX, TrainY, TestY)
  File "C:\Users\...\llmodel.py", line 65, in lstm
    model.fit(
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\tensorflow\python\keras\engine\training.py", line 1137, in fit
    data_handler = data_adapter.get_data_handler(
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 1394, in get_data_handler
    return DataHandler(*args, **kwargs)
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 1169, in __init__
    self._configure_dataset_and_inferred_steps(strategy, x, steps_per_epoch,
  File "C:\Users\...\AppData\Roaming\Python\Python310\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 1177, in _configure_dataset_and_inferred_steps
    if class_weight:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

How can this issue be solved?

CodePudding user response：

The log suggests that the problem is in your class_weight argument. According to the docs, this argument should be

Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to "pay more attention" to samples from an under-represented class.

which should be something like

class_weights = {0: 0.75, 1: 1.25}

if you want to assign a weight of 0.75 and 1.25 to class 0 and 1, respectively.

From the log, I suspect that instead of passing such a dictionary, you passed a numpy array of shape (n_samples, n_classes) as class_weight, which is inappropriate.