When training a tensorflow model with:

model.fit(..., steps_per_epoch=10000, ....)

I want to calculate the steps_per_epoch from the provided dataset:

dataset = tf.data.TFRecordDataset([filenames])
dataset = dataset.repeat(1)
dataset = dataset.batch(512)

total = 0
for i in dataset:
    total  = 1

print("Total is {}".format(total))

The output is:

Total is 393

Is the steps_per_epoch equal to 393?

or

steps_per_epoch = 393 / 512 ?

CodePudding user response：

The parameter steps_per_epoch is part of model training only when we use a large size dataset. steps_per_epoch determines the batches to train in a single dataset to improve the accuracy of the model.

steps_per_epoch depends on the batch_size and the training_set size

batch_size=50
trainingsize = 30000 

def calculate_spe(y, batch_size):
  return int(math.ceil((1. * y) / batch_size)) 

stepsPerEpoch = calculate_spe(trainingsize, batch_size)

after calculating steps per epochs you can use it as:

model.fit(..., steps_per_epoch = stepsPerEpoch , ....)