Why a part of photo is detected as a background-CodePudding

I have a piece of code that overlay face photo into frame. I have an issue that glassess are detected as background

What I have :

This is a picture that I want to overlay

But I got this result :

As you can see the problem connected with alpha channel and it's transperancy

My code is :


def fourChannels(img):
    height, width, channels = img.shape
    if channels < 4:
        new_img = cv2.cvtColor(img, cv2.COLOR_BGR2BGRA)
        return new_img
def transBg(img):   
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    th, threshed = cv2.threshold(gray, 187, 255, cv2.THRESH_BINARY_INV)
    # threshed= cv2.adaptiveThreshold(gray, 300, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)

    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (11,11))
    morphed = cv2.morphologyEx(threshed, cv2.MORPH_OPEN, kernel)
    roi, _ = cv2.findContours(morphed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)


    mask = np.zeros(img.shape, img.dtype)

    cv2.fillPoly(mask, roi, (255,)*img.shape[2], )
    masked_image = cv2.bitwise_and(img, mask)

    # masked_image = cv2.bitwise_and(img,dilated_img)
    return masked_image

CodePudding user response：

Your approach will not work since the threshold operation will involve the glasses as well.

As Micka suggested above, I found the largest external contour in the foreground and created a mask (1-channel) from it. This mask is now used as the alpha channel (transparency) for the foreground.

im = cv2.imread('face.png', cv2.IMREAD_UNCHANGED)
th = cv2.threshold(im[:,:,1], 50, 255, cv2.THRESH_BINARY)[1]

# find largest external contour
contours, hierarchy = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
c = max(contours, key = cv2.contourArea)

# create mask and draw largest contour
mask = np.zeros((im.shape[0], im.shape[1]), np.uint8)
mask = cv2.drawContours(mask,[c],0,255, -1)

im2 = im.copy()
# assign the mask to the alpha channel
im2[:,:,3] = mask

Now you can use im2 as an overlay on any background image.