Neural network on split data-CodePudding

I have images sample already split into 70% training and 30% testing

 #using this for question one with neural network
 originaldata_train, originaldata_test, targetoriginaldata_train,     targetoriginaldata_test = train_test_split(originalrepo, 
                                                                      target, test_size=0.3, 
                                                                      random_state=42, stratify=target)

  bindata_train, bindata_test, targetbindata_train, targetbindata_test = train_test_split(binarisedrepo, 
                                                                      target, test_size=0.3, 
                                                                      random_state=42, stratify=target)

I have both the binaries and original version split. and I want to apply neural network on one them.

I used keras

 model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(16,(3,3),activation = "relu" , input_shape = (180,180,3)) ,
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(32,(3,3),activation = "relu") ,  
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64,(3,3),activation = "relu") ,  
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(128,(3,3),activation = "relu"),  
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(), 
tf.keras.layers.Dense(550,activation="relu"),      #Adding the Hidden layer
tf.keras.layers.Dropout(0.1,seed = 2019),
tf.keras.layers.Dense(400,activation ="relu"),
tf.keras.layers.Dropout(0.3,seed = 2019),
tf.keras.layers.Dense(300,activation="relu"),
tf.keras.layers.Dropout(0.4,seed = 2019),
tf.keras.layers.Dense(200,activation ="relu"),
tf.keras.layers.Dropout(0.2,seed = 2019),
tf.keras.layers.Dense(5,activation = "softmax")   #Adding the Output Layer
])


from tensorflow.keras.optimizers import RMSprop,SGD,Adam
adam=Adam(lr=0.001)
model.compile(optimizer='adam',loss =     tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'])


history = model.fit(x_train,y_train,epochs = 500 , validation_data = (x_val, y_val))

But I got some errors

  <ipython-input-76-33734b1da1bc> in <module>()
  ----> 1 history = model.fit(x_train,y_train,epochs = 500 ,     validation_data = (x_val, y_val))

   1 frames
  /usr/local/lib/python3.7/dist-         packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
  1145           except Exception as e:  # pylint:disable=broad-   except
  1146             if hasattr(e, "ag_error_metadata"):
-> 1147               raise e.ag_error_metadata.to_exception(e)
   1148             else:
   1149               raise

ValueError: in user code:

File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1021, in train_function  *
    return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1010, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1000, in run_step  **
    outputs = model.train_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 859, in train_step
    y_pred = self(x, training=True)
File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
File "/usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py", line 264, in assert_input_compatibility
    raise ValueError(f'Input {input_index} of layer "{layer_name}" is '

ValueError: Input 0 of layer "sequential_5" is incompatible with the layer: expected shape=(None, 180, 180, 3), found shape=(None, 10000)

Any one with better approach to solve this or what I need to do please

CodePudding user response：

The error message is telling you the input data is in a wrong format. It doesn't seem the algorithm is wrong, but the data you are feeding it.

It expects an array of shape (180,180,3), a color image maybe? Whereas you are giving it some flat array with 10000 elements.

Double-check the input you are using; It is probably wrong.

I suspect you are giving the array of imageS (i.e, the list of image arrays, instead of the proper/individual array).

CodePudding user response：

The algorithm is fine but the issue is you are telling your algorithm to use shape (180, 180, 3) but you are feeding in shape 10000.

tf.keras.layers.Conv2D(16,(3,3),activation = "relu" , input_shape = (180,180,3)) ,

let calculate

180 * 180 * 3 = 97,200. which is not equals to 10,000.

now try this

Step

First convert your data into numpy array using

import numpy as np. np.array(originaldata_train)

try this to know the shape you can use

print(originaldata_train.shape).. this will give you a clue on size you can use . e.g (230,390,1)

which is the size you can use. you might want to be sure and still reset it again with

Remember you are reshaping with the values you print not 1852, 32, 1

originaldata_train = originaldata_train.reshape(1852, 32, 1)

Now let convert it to float

train_images = originaldata_train.astype('float32')

convert it here

train_images /= 255

You can now feed this to your algorithm

model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(16,(3,3),activation = "relu" , input_shape = 
  #this will be the shape you set above not 1852, 32, 1
( 1852, 32, 1)) ,
 tf.keras.layers.MaxPooling2D(2,2),

and try it again.

if you are stuck or need a reference, try this link Sample classification of Images with Neural Network