I'm working on a Neural Network and I've been training it recently, and it has approximately 93% accuracy on the training data and 0% accuracy on the validation data. My first thought was overfitting, but the model doesn't save in between training and I get these results in the first Epoch. I'm using keras in python with the following model code:
model = Sequential(
[
Conv1D(320, 8, input_shape=(560, 560), activation="relu"),
# Conv1D(320, 8, activation="relu"),
# Conv1D(320, 8, activation="relu"),
# Dense(750, activation="relu"),
# Dropout(0.6),
Dense(1500, activation="relu"),
Dropout(0.6),
Dense(750, activation="relu"),
Dropout(0.6),
GlobalMaxPooling1D(keepdims=True),
Dense(1, activation='softmax')
]
)
model.compile(optimizer=Adam(learning_rate=0.00001), loss="binary_crossentropy", metrics=['accuracy'])
earlystopping = callbacks.EarlyStopping(monitor="val_accuracy",
mode="max", patience=2,
restore_best_weights=True)
model1 = model.fit(x=training_x, y=training_y, batch_size=150, epochs=5, shuffle=True, verbose=1, callbacks=[earlystopping], validation_data=(val_x, val_y))
The results I'm getting look like this:
Epoch 1/5 167/167 [==============================] - 1266s 8s/step - loss: 6.4154 - accuracy: 0.9262 - val_loss: 0.0054 - val_accuracy: 0.0000e 00
I've tried changing almost all of the hyperparameters and changing the model's architecture but I keep getting similar results. Does this have anything to do with the data? The data I'm using is a 3d NumPy array containing pixel data from a bunch of images. Any help here would be greatly appreciated.
CodePudding user response:
You need to use activation='sigmoid'
and optimizers.RMSprop(lr=1e-4)
for a binary classification.