# load ResNet50
from tensorflow.keras.applications import ResNet50

resnet = ResNet50(
    include_top=True, # classification : True, embedding : False
    weights=None,
    input_shape=(32,32,3),
    pooling = 'avg', #gloval avarage pooling
    classes=10,
    classifier_activation ='softmax'
)

### resnet training strategy
from tensorflow.keras.callbacks import ReduceLROnPlateau

batch_size = 128
epochs = 30  # i used this Passively
lr = 0.1

optimizer = SGD(lr, momentum=0.9)
loss_fn = "sparse_categorical_crossentropy"
metrics = ["accuracy"]
lr_callback = ReduceLROnPlateau(
    monitor="val_loss",
    factor=0.1,
    patience=10,
    min_lr=1e-6
)

resnet.compile(optimizer=optimizer,
               loss=loss_fn,
               metrics=metrics)

# training ResNet50
resnet_history = resnet.fit(X_train, y_train,
                            batch_size=batch_size,
                            epochs=epochs,
                            validation_data=(X_test, y_test),
                            callbacks=[lr_callback],
                            verbose=1)

i have question to train with none-runtime-intialization and normal train e.g. i think 15 epoch is good movement. so, i did straight train 30 epoch not use runtime initialization. but it is ruined so next use runtime_initialization. i trained 45 epoch but is good. what is difference?

result graph

not use random_seed

15 epoch

15 30 epoch

45 epoch

-----------------answer add-------------------------------------

use random_seed not use ReduceLROnPlateau

15 epoch train

15 Subsequently 30 epoch(not use runtime initialize) train

45 epoch train

i don't know the difference. good movement of 45 epoch train but bad movement 15 30 epoch

CodePudding user response：

First, it may due to the randomness that affects more when the performance is not that good (Thus, large learning rate may hinder the model to converge. Try with 0.01 or 0.001).
To address this, I recommend to fix the seeds with the below code.

tf.set_random_seed(42)
random.seed(42) # optional 
numpy.random.seed(42) # optional

Moreover, learning rate or early stopping callbacks like ReduceLROnPlateau keep monitoring the model performance and save some values inside so that re-initializing may affect the result.