How to do string-style split on tf tensor with string values using TF-1.4-CodePudding

I have a tensor(e.g. labels) which read data from external data source. the value of the tensor is a string which has format "label1,label2" (e.g. "0,1"). Now I want to split the string values into a list using delimiter ',' so the result would be like ['0', '1'].
I've tried flowing:

# option-1 with error: tensor type has no attribute split.
label_list = input_label_tensor.split(',') 

# option-2 with error: module has no method split.
label_list = tf.strings.split(input_label_tensor, ',')

# option-3 with error: 
label_list = tf.string_split(input_label_tensor, ',')

the error from option-3 is:


  File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/string_ops.py", line 113, in string_split
    source = ops.convert_to_tensor(source, dtype=dtypes.string)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 836, in convert_to_tensor
    as_ref=False)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 926, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 774, in _TensorTensorConversionFunction
    (dtype.name, t.dtype.name, str(t)))
ValueError: Tensor conversion requested dtype string for Tensor with dtype float32: 'Tensor("StagingArea_get:467", shape=(?,), dtype=float32, device=/job:worker/task:0/device:CPU:0)'

what's the correct way to do the split operation? I'm using TF-1.4

CodePudding user response：

My best bet is that input_label_tensor is not actually a string tensor. For example this works:

%tensorflow_version 1.x
import tensorflow as tf

input_label_tensor = tf.constant(['0', '1'], dtype=tf.string)
label_list = tf.string_split(input_label_tensor, ',')

But if I use a float tensor, I am able to reproduce your error:

input_label_tensor = tf.constant([0, 1], dtype=tf.float32)
label_list = tf.string_split(input_label_tensor, ',')

To access the values of a SparseTensor try something like this:

%tensorflow_version 1.x
import tensorflow as tf

input_label_tensor = tf.constant(['0', '1'], dtype=tf.string)
label_list = tf.string_split(input_label_tensor, ',').values
x, y = label_list[0], label_list[1]

Update 1:

%tensorflow_version 1.x
import tensorflow as tf

input_label_tensor = tf.constant(['0,1', '1,0', '1,1', '0,0'], dtype=tf.string)
label_list = tf.string_split(input_label_tensor, ',')

c = tf.sparse.to_dense(label_list)
c = tf.string_to_number(c, tf.float32)

with tf.Session() as sess:
    result = c.eval()
   
print(result)

[[0. 1.]
 [1. 0.]
 [1. 1.]
 [0. 0.]]