Keras Neural Network With Custom Input-CodePudding

What I'm trying to accomplish is having a set relationship between some custom input data and output data then have a neural network figure out this relationship/rule to predict future output given the input. I've set up some test code here where a random list of inputs is generated and if it is more than 0.5 the output is 1 otherwise the output is 0.

from tensorflow import keras
import numpy as np

# generate data
data_input_generate = np.random.random((6400, 1))
data_output_generate = np.random.randint(2, size=(6400, 1))
data_input = np.vstack([data_input_generate, data_input_generate])
data_output = np.vstack([data_output_generate, data_output_generate])

for i in range(len(data_input)):

    if data_input[i] >= 0.5:
        data_output[i] = [1]
    else:
        data_output[i] = [0]


# setup neural network
Inputs = keras.layers.Input(shape=(1, ))
hidden1 = keras.layers.Dense(units=100, activation="sigmoid")(Inputs)
hidden2 = keras.layers.Dense(units=100, activation='softmax')(hidden1)
predictions = keras.layers.Dense(units=1, activation='relu')(hidden2)

# initialize model
model = keras.Model([Inputs], outputs=predictions)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# fit
model.fit(data_input, data_output, batch_size=10, epochs=5)

# predict
predictions = model.predict(data_input_generate)

# print predictions
for i in range(10):
    print(f"Value: {data_input_generate[i]}, Result: {data_output_generate[i]}, Prediction: {predictions[i]}")

The problem is, after fitting the model, the accuracy stays at 50%. Is this a problem with my layer activation function or the way I set up the model? My goal is to correctly predict the output with fairly high accuracy. Thanks in advance!

CodePudding user response：

Try using a sigmoid activation function on your output layer. Here is a working example:

from tensorflow import keras
import numpy as np

# generate data
data_input_generate = np.random.random((6400, 1))
data_output_generate = np.random.randint(2, size=(6400, 1))
data_input = np.vstack([data_input_generate, data_input_generate])
data_output = np.vstack([data_output_generate, data_output_generate])

for i in range(len(data_input)):

    if data_input[i] >= 0.5:
        data_output[i] = [1]
    else:
        data_output[i] = [0]


# setup neural network
Inputs = keras.layers.Input(shape=(1, ))
hidden1 = keras.layers.Dense(units=64, activation="relu")(Inputs)
hidden2 = keras.layers.Dense(units=32, activation='relu')(hidden1)
dropout = keras.layers.Dropout(0.8)(hidden2)
predictions = keras.layers.Dense(units=1, activation='sigmoid')(dropout)

# initialize model
model = keras.Model([Inputs], outputs=predictions)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# fit
model.fit(data_input, data_output, batch_size=10, epochs=5)

# predict
predictions = model.predict(data_input_generate)

# print predictions
for i in range(10):
    print(f"Value: {data_input_generate[i]}, Result: {data_output_generate[i]}, Prediction: {predictions[i]}")

Epoch 1/5
1280/1280 [==============================] - 4s 3ms/step - loss: 0.3632 - accuracy: 0.8327
Epoch 2/5
1280/1280 [==============================] - 3s 3ms/step - loss: 0.1870 - accuracy: 0.9427
Epoch 3/5
1280/1280 [==============================] - 3s 3ms/step - loss: 0.1528 - accuracy: 0.9475
Epoch 4/5
1280/1280 [==============================] - 3s 2ms/step - loss: 0.1461 - accuracy: 0.9482
Epoch 5/5
1280/1280 [==============================] - 2s 2ms/step - loss: 0.1384 - accuracy: 0.9493
Value: [0.79415764], Result: [0], Prediction: [0.9997529]
Value: [0.38311113], Result: [1], Prediction: [1.7478478e-05]
Value: [0.05360975], Result: [0], Prediction: [2.3240638e-07]
Value: [0.78635261], Result: [1], Prediction: [0.99970365]
Value: [0.74414175], Result: [1], Prediction: [0.99921006]
Value: [0.47845171], Result: [1], Prediction: [0.07256863]
Value: [0.53008247], Result: [0], Prediction: [0.886382]
Value: [0.40377478], Result: [1], Prediction: [9.9769844e-05]
Value: [0.18209166], Result: [1], Prediction: [5.199377e-07]
Value: [0.00937745], Result: [1], Prediction: [1.7613968e-07]