record the computation time for each epoch during model.fit()-CodePudding

I am using the standard keras process to train a model:

model.compile(optimizer = 'adam', loss = 'mean_squared_error')
model.fit(x_train, y_train, epochs = 50)

And the output is something like:

Epoch 1/50
52/52 [==============================] - 8s 90ms/step - loss: 0.0027
Epoch 2/50
52/52 [==============================] - 5s 90ms/step - loss: 0.0023

How can I calculate elapsed time for each Epoch?

I will have more than one experiment so I want to compare times.

CodePudding user response：

The proper way to do it, would be to write your own training callback inheriting from the Callback class.

You then need to override the methods on_epoch_begin() and on_epoch_end() which are called by model.fit() during training. There you can start and end counters, and save the results to internal variables to access them later.

But wouldn't it be easier to simple measure the time of the whole training process and divide by the number of epochs? (Although, that would also take into account the validation time)

CodePudding user response：

You need to create a simple custom callback. Documentation for that is here. My code to do that is shown below

import time
class etimer(keras.callbacks.Callback):
    def __init__ (self): # initialization of the callback
        super(etimer, self).__init__()
    def on_epoch_begin(self,epoch, logs=None):
        self.now= time.time()
    def on_epoch_end(self,epoch, logs=None): 
        later=time.time()
        duration=later-self.now 
        print('\nfor epoch ', epoch  1, ' the duration was ', duration, ' seconds')

in model.fit add callbacks=callbacks