How can I efficiently concatenate images in a Tensorflow dataset?-CodePudding

I currently have sixteen images (A,B,C,D,E,F,G,...) which must be concatenated into one as part of a Tensorflow Dataset workflow. Each image is 128 x 128 and has the shape of (128, 128, 3). The final output should be a 512 x 512 image of shape (512,512,3). All of the images come from an image sequence, known as img_seq.

Right now, this is accomplished through the following code:

@tf.function
def glue_to_one(imgs_seq):
    first_row= tf.concat((imgs_seq[0], imgs_seq[1],imgs_seq[2],imgs_seq[3]), 0)
    second_row = tf.concat((imgs_seq[4], imgs_seq[5], imgs_seq[6], imgs_seq[7]), 0)
    third_row = tf.concat((imgs_seq[8], imgs_seq[9], imgs_seq[10], imgs_seq[11]), 0)
    fourth_row = tf.concat((imgs_seq[12], imgs_seq[13], imgs_seq[14], imgs_seq[15]), 0)

    img_glue = tf.stack((first_row, second_row, third_row, fourth_row), axis=1)
    img_glue = tf.reshape(img_glue, [512,512,3])

    return img_glue

It is suspected that this method is inefficient and is learning to a bottleneck. A different approach would be to allocate a 512 x 512 tensor and then replace the elements. Would this be more efficient? How would it be done? Can you please recommend a better approach?

CodePudding user response：

You can improve it about 3 times using something like this:

def glue_answer(imgs_seq):
    image = tf.reshape(imgs_seq, (4, 4, 128, 128, 3))
    image = tf.concat(image, axis=1)
    image = tf.concat(image, axis=1)
    
    return image

I tested the performance as follows:

def glue_to_one(imgs_seq):
    first_row= tf.concat((imgs_seq[0], imgs_seq[1],imgs_seq[2],imgs_seq[3]), 0)
    second_row = tf.concat((imgs_seq[4], imgs_seq[5], imgs_seq[6], imgs_seq[7]), 0)
    third_row = tf.concat((imgs_seq[8], imgs_seq[9], imgs_seq[10], imgs_seq[11]), 0)
    fourth_row = tf.concat((imgs_seq[12], imgs_seq[13], imgs_seq[14], imgs_seq[15]), 0)

    img_glue = tf.stack((first_row, second_row, third_row, fourth_row), axis=1)
    img_glue = tf.reshape(img_glue, [512,512,3])

    return img_glue

def glue_answer(imgs_seq):
    image = tf.reshape(imgs_seq, (4, 4, 128, 128, 3))
    image = tf.concat(image, axis=1)
    image = tf.concat(image, axis=1)
    
    return image

print("Method in question:")
%timeit -n 1000 -r 10 glue_to_one(imgs_seq)
print("Method in answe:")
%timeit -n 1000 -r 10 glue_answer(imgs_seq)

Output:

Method in question:
1.7 ms ± 212 µs per loop (mean ± std. dev. of 10 runs, 1,000 loops each)
Method in answe:
540 µs ± 28.8 µs per loop (mean ± std. dev. of 10 runs, 1,000 loops each)

CodePudding user response：

Simply use tf.split method instead of writing that much code...,

tensor = tf.concat(tf.split(x, 4, axis=0), axis=1)

Output-Shape:

<tf.Tensor: shape=(4, 512, 128, 3), dtype=float32, numpy=

Now, use reshape Method...

tf.reshape(tensor, (512, 512, 3))

Output:

tf.Tensor: shape=(512, 512, 3), dtype=float32, numpy