How to dilate y_true inside a custom metric in keras/tensorflow?-CodePudding

I am trying to code a custom metric for U-net model implemented using keras/tensorflow. In the metric, I need to use the opencv function, 'cv2.dilate' on the ground truth. When I tried to use it, it gave the error as y_true is a tensor and cv2.dilate expects a numpy array. Any idea on how to implement this?

I tried to convert tensor to numpy array but it is not working. I searched for the tensorflow implementation of cv2.dilate but couldnt find one.

CodePudding user response：

One possibility, if you are using a simple rectangular kernel in your dilation, is to use tf.nn.max_pool2d as a replacement.

import numpy as np
import tensorflow as tf
import cv2

image = np.random.random((28,28))
kernel_size = 3
# OpenCV dilation works with grayscale image, with H,W dimensions
dilated_cv = cv2.dilate(image, np.ones((kernel_size, kernel_size), np.uint8))
# TensorFlow maxpooling works with batch and channels: B,H,W,C dimenssions
image_w_batch_and_channels = image[None,...,None]
dilated_tf = tf.nn.max_pool2d(image_w_batch_and_channels, kernel_size, 1, "SAME")

# checking that the results are equal
np.allclose(dilated_cv, dilated_tf[0,...,0])

However, given that you mention that you are applying dilation on the ground truth, this dilation does not need to be differentiable. In that case, you can wrap your dilation in a tf.numpy_function

from functools import partial
# be sure to put the correct output type, tf.float64 is working in that specific case because numpy defaults to float64, but it might be different in your case
dilated_tf_npfunc = tf.numpy_function(
    partial(cv2.dilate, kernel=np.ones((kernel_size, kernel_size), np.uint8)), [image]
)