how to design tf.keras callback to save model predictions for each batch and each epoch-CodePudding

I want to create a tf.keras callback to save model predictions for each batch and each epoch during the training

i have tried the following callback, however it gives error like

AttributeError: 'PredictionCallback' object has no attribute 'X_train'

My code is

class PredictionCallback(tf.keras.callbacks.Callback):    

  def on_epoch_end(self, epoch, logs={}):

    y_pred = self.model.predict(self.X_train)

    print('prediction: {} at epoch: {}'.format(y_pred, epoch))

    pd.DataFrame(y_pred).assign(epoch=epoch).to_csv('{}_{}.csv'.format(filename, epoch))

    cnn_model.fit(X_train, y_train,validation_data=[X_valid,y_valid],epochs=epochs,batch_size=batch_size,
               callbacks=[model_checkpoint,reduce_lr,csv_logger, early_stopping,PredictionCallback()],
               verbose=1)

i also tried Create keras callback to save model predictions and targets for each batch during training but not get success yet.Hope experts will help me.Thanks.

CodePudding user response：

I tried

class PredictionCallback(tf.keras.callbacks.Callback):    
  def on_epoch_end(self, epoch, logs={}):
    y_pred = self.model.predict(self.validation_data[0])
    print('prediction: {} at epoch: {}'.format(y_pred, epoch))
    pd.DataFrame(y_pred.reshape(200,80)).assign(epoch=epoch).to_csv('{}_{}.csv'.format('filename', epoch))
    np.savetxt('output.txt',y_pred.reshape(200,80))

But it doesnot save the results in filename, why?

CodePudding user response：

Hello you are on the right track. You can store it via a txt file using the following callback function:

class PredictionCallback(tf.keras.callbacks.Callback): 
  def __init__(self, model, test_data):
    self.model = model
    self.test_data = test_data
    
  def on_epoch_end(self, epoch, logs={}):

    x,y = self.test_data
    y_pred = self.model.predict(x)

Afterwards you can train your model using tensorflow's fit function:


history = model.fit(x1, y1, batch_size=128, epochs=10,
                    callbacks= [PredictionCallback(model, [x2, y2] )])

after previously having defined your architecture:

model = network()

This worked for me. See if you also are in the correct path of your folder.