ValueError: invalid literal for int() with base 16: 'Interstitial'-CodePudding

I want to convert the below string into categorical form or one hot encoded.

string1 = "Interstitial markings are diffusely prominent throughout both lungs. Heart size is normal. Pulmonary XXXX normal."
st1 = string1.split()

I am using below code but it generates error.

from numpy import array
from numpy import argmax
from keras.utils import to_categorical
# define example
data = array(st1)
print(data)
encoded = to_categorical(data)
print(encoded)
# invert encoding
inverted = argmax(encoded[0])
print(inverted)

error

['Interstitial' 'markings' 'are' 'diffusely' 'prominent' 'throughout' 'both' 'lungs.' 'Heart' 'size' 'is' 'normal.' 'Pulmonary' 'XXXX''normal.']
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-b034d9393342> in <module>
  5 data = array(st1)
  6 print(data)
----> 7 encoded = to_categorical(data)
  8 print(encoded)
  9 # invert encoding

/usr/local/lib/python3.7/dist-packages/keras/utils/np_utils.py in to_categorical(y, num_classes, dtype)
 60   [0. 0. 0. 0.]
 61   """
---> 62   y = np.array(y, dtype='int')
 63   input_shape = y.shape
 64   if input_shape and input_shape[-1] == 1 and len(input_shape) > 1:

ValueError: invalid literal for int() with base 10: 'Interstitial'

CodePudding user response：

Logically error wise says you typecasting str to int.

Like int('20') = 20 - Correct

int('Interstitial') - ValueError: invalid literal for int() with base 16: 'Interstitial'

This is because

keras only supports one-hot-encoding for data that has already been integer-encoded.

In such cases you can do so use LabelEncoder as follows.

string1 = "Interstitial markings are diffusely prominent throughout both lungs. Heart size is normal. Pulmonary XXXX normal."
st1 = string1.split()
from sklearn.preprocessing import LabelEncoder
import numpy as np

data = np.array(st1)

label_encoder = LabelEncoder()
data = label_encoder.fit_transform(data)
print(data)
##
##
##From here encode according next part of your code using to_categorical(data)

Gives #

array([ 1,  9,  4,  6, 11, 13,  5,  8,  0, 12,  7, 10,  2,  3, 10],
      dtype=int64)

CodePudding user response：

Tensorflow has clearly mentioned it here that the tf.keras.utils.to_categorical is for converting a class vector (integers) to binary class matrix.

Your data variable contains string type elements, which is not same as integer, hence the error.