Getting value error for Concatenate layer-CodePudding

Well, I'm trying to replicate a model from this

well, I've made some progress I tried my best to replicate the paper but in the end, I'm getting a value error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-55-d583261b0496> in <module>()
     23 
     24 # 4. create an output layer
---> 25 dropout_layer = tf.keras.layers.Dropout(0.5,name='dropout_layer')(concat_model)
     26 dense_layer = tf.keras.layers.Dense(200,name='dense_layer')(dropout_layer)
     27 dropout_layer = tf.keras.layers.Dropout(0.5,name='dropout_layer_2')(dense_layer)

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py in convert_to_eager_tensor(value, ctx, dtype)
    100       dtype = dtypes.as_dtype(dtype).as_datatype_enum
    101   ctx.ensure_initialized()
--> 102   return ops.EagerTensor(value, ctx.device_name, dtype)
    103 
    104 

ValueError: Exception encountered when calling layer "dropout_layer" (type Dropout).

Attempt to convert a value (<keras.layers.merge.Concatenate object at 0x7f51a0b5f850>) with an unsupported type (<class 'keras.layers.merge.Concatenate'>) to a Tensor.

Call arguments received:
  • inputs=<keras.layers.merge.Concatenate object at 0x7f51a0b5f850>
  • training=False

for the code, I tried my best to comment on as many things as possible.

# 1. word model:
inputs = tf.keras.layers.Input(
    shape=(1,), dtype=tf.string, name="text_input"
)  # takes a list of input
text_vectorization_layer = text_vectorizer(inputs)  # word level vectorizer
text_embedding_layer = embedding(text_vectorization_layer)  # word level embedding
# flatten_layer = tf.keras.layers.Flatten()(text_embedding_layer)
outputs = tf.keras.layers.Dense(300, activation="relu", name="text_output")(
    text_embedding_layer
)  # output of 300 from fig 1. https://arxiv.org/pdf/1612.05251.pdf
token_model = tf.keras.Model(inputs, outputs)  # token model

# 2. char model:
inputs = tf.keras.layers.Input(shape=(1,), dtype=tf.string, name="char_input")
char_vectorization_layer = char_vectorizer(inputs)  # character level vectorization
char_embedding_layer = char_embedding(char_vectorization_layer)  # char level embedding
outputs = tf.keras.layers.Bidirectional(
    tf.keras.layers.LSTM(25), name="bi_directional_lstm"
)(
    char_embedding_layer
)  # bi directional lstm as output
char_model = tf.keras.Model(inputs, outputs)

# 3. concatinating both the model
concat_model = tf.keras.layers.Concatenate([token_model, char_model])

# 4. create output layer
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer")(concat_model)
dense_layer = tf.keras.layers.Dense(200, name="dense_layer")(dropout_layer)
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer_2")(dense_layer)
output_layer = tf.keras.layer.Dense(5, activation="softmax", name="final_output_layer")(
    dropout_layer
)

I'm not getting what will be the output from the concatenate layer that it's causing an error for the dropout layer

fixed code thanks to @Djinn :

# 1. word model:
inputs = tf.keras.layers.Input(
    shape=(1,), dtype=tf.string, name="text_input"
)  # takes a list of input
text_vectorization_layer = text_vectorizer(inputs)  # word level vectorizer
text_embedding_layer = embedding(text_vectorization_layer)  # word level embedding
flatten_layer = tf.keras.layers.Flatten()(text_embedding_layer)
outputs = tf.keras.layers.Dense(300, activation="relu", name="text_output")(
    flatten_layer
)  # output of 300 from fig 1. https://arxiv.org/pdf/1612.05251.pdf
token_model = tf.keras.Model(inputs, outputs)  # token model

# 2. char model:
inputs = tf.keras.layers.Input(shape=(1,), dtype=tf.string, name="char_input")
char_vectorization_layer = char_vectorizer(inputs)  # character level vectorization
char_embedding_layer = char_embedding(char_vectorization_layer)  # char level embedding
outputs = tf.keras.layers.Bidirectional(
    tf.keras.layers.LSTM(25), name="bi_directional_lstm"
)(
    char_embedding_layer
)  # bi directional lstm as output
char_model = tf.keras.Model(inputs, outputs)

# 3. concatinating both the model
concat_model = tf.keras.layers.Concatenate()([token_model.output, char_model.output])

# 4. create output layer
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer")(concat_model)
dense_layer = tf.keras.layers.Dense(200, name="dense_layer")(dropout_layer)
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer_2")(dense_layer)
output_layer = tf.keras.layers.Dense(
    5, activation="softmax", name="final_output_layer"
)(dropout_layer)

# 5.final model
model_2 = tf.keras.Model(
    inputs=[token_model.input, char_model.input],
    outputs=output_layer,
    name="token_char_model",
)

CodePudding user response：

You are concatenating two models. You probably want to concatenate their outputs (or the last layer if you wanted to do it that way. You can concatenate any compatible layers if you wanted). You're also calling Concatenate incorrectly. Concatenate with a capital C is Concatenate(params)([layers]). concatenate with lowercase c is concatenate([layers], params)

Change:

concat_model = tf.keras.layers.Concatenate([token_model, char_model])

concat_model = tf.keras.layers.Concatenate()([token_model.output, char_model.output])

# or if you changed the name of the two outputs to, for example, [output0, output1]
concat_model = tf.keras.layers.Concatenate()([output0, output1])