Specific options missing in keras layer class-CodePudding

I would like to implement operations on the results of two keras conv2d layers (Ix,Iy) in a deep learning architecture for a computer vision task. The operation looks as follows:

G = np.hypot(Ix, Iy)
G = G / G.max() * 255
theta = np.arctan2(Iy, Ix)

I've spent some time looking for operations provided by keras but did not have success so far. Among a few others, there's a "add" functionality that allows the user to add the results of two conv2d layers (tf.keras.layers.Add(Ix,Iy)). However, I would like to have a Pythagorean addition (first line) followed by a arctan2 operation (third line).

So ideally, if already implemented by keras it would look as follows:

tf.keras.layers.Hypot(Ix,Iy)   
tf.keras.layers.Arctan2(Ix,Iy)

Does anyone know if it is possible to implement those functionalities within my deep learning architecture? Is it possible to write custom layers that meet my needs?

CodePudding user response：

You could probably use simple Lambda layers for your use case, although they are not absolutely necessary:

import tensorflow as tf

inputs = tf.keras.layers.Input((16, 16, 1))
x = tf.keras.layers.Conv2D(32, (3, 3), padding='same')(inputs)
y = tf.keras.layers.Conv2D(32, (2, 2), padding='same')(inputs)
hypot = tf.keras.layers.Lambda(lambda z: tf.math.sqrt(tf.math.square(z[0])   tf.math.square(z[1])))([x, y])
hypot = tf.keras.layers.Lambda(lambda z: z / tf.reduce_max(z) * 255)(hypot)
atan2 = tf.keras.layers.Lambda(lambda z: tf.math.atan2(z[0], z[1]))([x, y])

model = tf.keras.Model(inputs, [hypot, atan2])
print(model.summary())

model.compile(optimizer='adam', loss='mse')

model.fit(tf.random.normal((64, 16, 16, 1)), [tf.random.normal((64, 16, 16, 32)), tf.random.normal((64, 16, 16, 32))])

Model: "model_1"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_3 (InputLayer)           [(None, 16, 16, 1)]  0           []                               
                                                                                                  
 conv2d_2 (Conv2D)              (None, 16, 16, 32)   320         ['input_3[0][0]']                
                                                                                                  
 conv2d_3 (Conv2D)              (None, 16, 16, 32)   160         ['input_3[0][0]']                
                                                                                                  
 lambda_2 (Lambda)              (None, 16, 16, 32)   0           ['conv2d_2[0][0]',               
                                                                  'conv2d_3[0][0]']               
                                                                                                  
 lambda_3 (Lambda)              (None, 16, 16, 32)   0           ['lambda_2[0][0]']               
                                                                                                  
 lambda_4 (Lambda)              (None, 16, 16, 32)   0           ['conv2d_2[0][0]',               
                                                                  'conv2d_3[0][0]']               
                                                                                                  
==================================================================================================
Total params: 480
Trainable params: 480
Non-trainable params: 0
__________________________________________________________________________________________________
None
2/2 [==============================] - 1s 71ms/step - loss: 3006.0469 - lambda_3_loss: 3001.7981 - lambda_4_loss: 4.2489
<keras.callbacks.History at 0x7ffa93dc2890>