Read image labels from a csv file-CodePudding

I have a dataset of medical images (.dcm) which I can read into TensorFlow as a batch. However, the problem that I am facing is that the labels of these images are in a .csv. The .csv file contains two columns - image_path (location of the image) and image_labels (0 for no; 1 for yes). I wanted to know how I can read the labels into a TensorFlow dataset batch wise. I am using the following code to load the images batch wise:-

import tensorflow as tf
import tensorflow_io as tfio

def process_image(filename):
    image_bytes = tf.io.read_file(filename)
    image = tf.squeeze(
        tfio.image.decode_dicom_image(image_bytes, on_error='strict', dtype=tf.uint16),
        axis = 0
    )
    x = tfio.image.decode_dicom_data(image_bytes, tfio.image.dicom_tags.PhotometricInterpretation)
    image = (image - tf.reduce_min(image))/(tf.reduce_max(image) - tf.reduce_min(image))
    if(x == "MONOCHROME1"):
        image = 1 - image
    image = image*255
    image = tf.cast(tf.image.resize(image, (512, 512)),tf.uint8)
    return image

# train_images is a list containing the locations of .dcm images
dataset = tf.data.Dataset.from_tensor_slices(train_images)
dataset = dataset.map(process_image, num_parallel_calls=4).batch(50)

Hence, I can load the images into the TensorFlow dataset. But I would like to know how I can load the image labels batch wise.

CodePudding user response：

Something like this instead of the last two lines should work:

#train_labels is a list of labels for each image in the same order as in train_images

dataset = tf.data.Dataset.from_tensor_slices((train_images, train_labels))
dataset = dataset.map(lambda x,y : (process_image(x), y), num_parallel_calls=4).batch(50)

now the dataset can be passed to your network's .fit(), .predict() and other methods:

model.fit(dataset, epochs=epochs, callbacks=callbacks)

Alternatively, you can create a second dataset containing the labels and then combine two datasets with tf.data.Dataset.zip(). It works similarly to the python's native zip.

I prefer the first method since It feels a bit cleaner to me I can, for example, shuffle the filenames/labels and only then parse the files instead of doing the opposite.