The problem I have is the same as Neural network with a single out with tensorflow
I have a problem classifying between two classes. My data is labeled by 0s and 1s. I want to use Tensorflow neural network with one node at the output, so the result will be the probability between 0 and 1 in the example from the ones. Here is my code attempt:
example_size = 100
X = tf.random.normal((example_size,2))
y = tf.constant([[int(x)] for x in (X[:,0] > X[:,1])])
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape = (2,)))
model.add(tf.keras.layers.Dense(units = 1 , activation = "sigmoid")) # output layer
loss = tf.keras.losses.SparseCategoricalCrossentropy()
optim = tf.keras.optimizers.Adam(learning_rate = 0.01)
metrics = ["accuracy"]
model.compile(loss = loss, optimizer = optim, metrics = metrics)
model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)
The error code:
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
Input In [38], in <cell line: 11>()
9 metrics = ["accuracy"]
10 model.compile(loss = loss, optimizer = optim, metrics = metrics)
---> 11 model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)
File c:\python\python39\lib\site-packages\keras\utils\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File c:\python\python39\lib\site-packages\tensorflow\python\eager\execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
52 try:
53 ctx.ensure_initialized()
---> 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
55 inputs, attrs, num_outputs)
56 except core._NotOkStatusException as e:
57 if name is None:
InvalidArgumentError: Graph execution error:
Detected at node 'sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits' defined at (most recent call last):
File "C:\Users\Lior\anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Lior\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel_launcher.py", line 16, in <module>
app.launch_new_instance()
File "C:\Users\Lior\anaconda3\lib\site-packages\traitlets\config\application.py", line 846, in launch_instance
app.start()
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 677, in start
self.io_loop.start()
File "C:\Users\Lior\anaconda3\lib\site-packages\tornado\platform\asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "C:\Users\Lior\anaconda3\lib\asyncio\base_events.py", line 601, in run_forever
self._run_once()
File "C:\Users\Lior\anaconda3\lib\asyncio\base_events.py", line 1905, in _run_once
handle._run()
File "C:\Users\Lior\anaconda3\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 471, in dispatch_queue
await self.process_one()
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 460, in process_one
await dispatch(*args)
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 367, in dispatch_shell
await result
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 662, in execute_request
reply_content = await reply_content
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 360, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 532, in run_cell
return super().run_cell(*args, **kwargs)
File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2863, in run_cell
result = self._run_cell(
File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2909, in _run_cell
return runner(coro)
File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3106, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3309, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3369, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "C:\Users\Lior\AppData\Local\Temp\ipykernel_27496\3786262486.py", line 11, in <cell line: 11>
model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)
File "c:\python\python39\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1409, in fit
tmp_logs = self.train_function(iterator)
File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1051, in train_function
return step_function(self, iterator)
File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1040, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1030, in run_step
outputs = model.train_step(data)
File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 890, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 948, in compute_loss
return self.compiled_loss(
File "c:\python\python39\lib\site-packages\keras\engine\compile_utils.py", line 201, in __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "c:\python\python39\lib\site-packages\keras\losses.py", line 139, in __call__
losses = call_fn(y_true, y_pred)
File "c:\python\python39\lib\site-packages\keras\losses.py", line 243, in call
return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "c:\python\python39\lib\site-packages\keras\losses.py", line 1860, in sparse_categorical_crossentropy
return backend.sparse_categorical_crossentropy(
File "c:\python\python39\lib\site-packages\keras\backend.py", line 5238, in sparse_categorical_crossentropy
res = tf.nn.sparse_softmax_cross_entropy_with_logits(
Node: 'sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits'
Received a label value of 1 which is outside the valid range of [0, 1). Label values: 0 0 1 1 1 0 1 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 0 1 0 1 0 0 0 1 0 0 1 1 0 0 1 0 0 1 1 1 1 0 0 1 0 0 0 0 1 0 1 0 1 0 0 1 0 0 0 0 0 1 1 1 1 1 1 0
[[{{node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits}}]] [Op:__inference_train_function_15984]
How I can fix it?
I also read the comment to Neural network with a single out with tensorflow witch is led me to How to choose cross-entropy loss in TensorFlow? , but that question is not about my topic at all. And I already used the sigmoid cost that he recommended in the binary case.
CodePudding user response：
Use the binary cross entropy loss function for binary classification tasks:
import tensorflow as tf
example_size = 100
X = tf.random.normal((example_size,2))
y = tf.constant([[int(x)] for x in (X[:,0] > X[:,1])])
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape = (2,)))
model.add(tf.keras.layers.Dense(units = 1 , activation = "sigmoid")) # output layer
loss = tf.keras.losses.BinaryCrossentropy()
optim = tf.keras.optimizers.Adam(learning_rate = 0.01)
metrics = ["accuracy"]
model.compile(loss = loss, optimizer = optim, metrics = metrics)
model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)
If you really want to use tf.keras.losses.SparseCategoricalCrossentropy, change your output layer to:
model.add(tf.keras.layers.Dense(units = 2 , activation = "softmax")) # output layer
See this post for more information.