Reshaping problem (Input to reshape is a tensor with 10 values, but the requested shape has 1)-CodePudding

I'm trying to recreate this work using my own dataset: https://www.kaggle.com/code/amyjang/tensorflow-pneumonia-classification-on-x-rays/notebook

I've made some slight tweaks to the code to accommodate my data but I don't think that is what is causing an issue here; it could be though of course.

My code:

AUTOTUNE = tf.data.AUTOTUNE
BATCH_SIZE = 16
IMAGE_SIZE = [180, 180]
EPOCHS = 25
CLASS_NAMES = np.array(['active', 'inactive'])

train_list_ds = tf.data.Dataset.from_tensor_slices(glob.glob(f'{WORKING}/train/*/*'))
val_list_ds = tf.data.Dataset.from_tensor_slices(glob.glob(f'{WORKING}/val/*/*'))
test_list_ds = tf.data.Dataset.from_tensor_slices(glob.glob(f'{WORKING}/test/*/*'))

#print(next(train_list_ds.batch(60_000).as_numpy_iterator())[:5])

def get_label(path):
  # switcher that converts the keywords to 1 or zero
  if tf.strings.split(path, os.path.sep) == "inactive":
    return 0 # inactive
  else:
    return 1 # active

def decode_img(img):
  # convert the compressed string to a 3D uint8 tensor
  img = tf.image.decode_jpeg(img, channels=3)
  # Use `convert_image_dtype` to convert to floats in the [0,1] range.
  img = tf.image.convert_image_dtype(img, tf.float32)
  # resize the image to the desired size.
  resizedImage = tf.image.resize(img, IMAGE_SIZE)
  return resizedImage

def process_path(file_path):
    label = get_label(file_path)
    # load the raw data from the file as a string
    img = tf.io.read_file(file_path)
    img = decode_img(img)
    return img, label

train_ds = train_list_ds.map(process_path, num_parallel_calls=AUTOTUNE)
val_ds = val_list_ds.map(process_path, num_parallel_calls=AUTOTUNE)
test_ds = test_list_ds.map(process_path, num_parallel_calls=AUTOTUNE)
test_ds = test_ds.batch(BATCH_SIZE)

image_batch, label_batch = next(iter(train_ds))

And the error:

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-21-d0b00c7e96e2> in <module>
     68 test_ds = test_ds.batch(BATCH_SIZE)
     69 
---> 70 image_batch, label_batch = next(iter(train_ds))
     71 
     72 # Use buffered prefetching so we can yield data from disk without having I/O become blocking.

3 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
   7184 def raise_from_not_ok_status(e, name):
   7185   e.message  = (" name: "   name if name is not None else "")
-> 7186   raise core._status_to_exception(e) from None  # pylint: disable=protected-access
   7187 
   7188 

InvalidArgumentError: Input to reshape is a tensor with 10 values, but the requested shape has 1
     [[{{node Reshape}}]] [Op:IteratorGetNext]

I can gather from the error that I have a mismatch in resizing, I believe it is pointing to the wrong place though and it is something to do with the line resizedImage = tf.image.resize(img, IMAGE_SIZE)

I've looked through the docs and on SO but I can't find anything that might help me. I've tried debugging as best I can by printing things out on screen. I also tried changing the IMAGE_SIZE but that made no difference.

On the images, they are different sizes on disk, in JPG form. I expected that not to matter as we can use this step tot resize them for processing by the model later.

Other useful information is that I'm working on the pro version of Google Collab and the files are stored in a Google drive location. Again I don't think that's a problem or cause of anything; I'm trying to write things pre-emptively that people might ask.

Finally there is a prepare_for_training function in Amy's code on Kaggle that is called before the last line I provided in my code above. I can trigger the same error without calling that function so I omitted it intentionally to help keep the example code cleaner. If you want to see it, here is a quick link to it in the notebook: https://www.kaggle.com/code/amyjang/tensorflow-pneumonia-classification-on-x-rays?scriptVersionId=39162263&cellId=26

CodePudding user response：

May be the problem come from the get_label(path) method, since you are splitting path according the sep, it will return a list of many elements, make sure that you select only one element to perform test, try this :

tf.strings.split(path, os.path.sep)[-1] == "inactive"

i assume that the last element is the label, you just replace this index by the position of the label in this list.