I'm trying to implement an image captioning model but got at the first epoch this result

  - loss: 3.9312 - accuracy: 0.3160 - val_loss: 135.0646 - val_accuracy: 0.0247

the model is

def define_model(vocab_size,max_length):
    
    inputs1=Input(shape=(512,))
    
    #fe1 = Dropout(0.5)(inputs1)
    
    fe2 = Dense(256, activation='relu')(fe1)

    inputs2 = Input(shape=(max_length,))

    se1 = Embedding(vocab_size, 256, mask_zero=True)(inputs2)

    se4 = LSTM(256)(se1)

    decoder1 = Concatenate()([fe2, se4])

    decoder2 = Dense(256, activation='relu')(decoder1)

    outputs = Dense(vocab_size, activation='softmax')(decoder2)
    
    model=Model(inputs=[inputs1,inputs2],outputs=outputs)

    model.compile(loss='categorical_crossentropy',optimizer='adam', metrics=['accuracy'])
    
    return model

what may cause this problem ? why val_loss much increase than the loss

CodePudding user response：

Try these possible solutions mate

1. Reduce your network size
2. Remove Dropout layer from that position and place after input layer Hope this works.