how to crop an image when detecting a straight line in python-CodePudding

I am planning to build an OCR (Optical character recognition), and now I am cleaning the data.

I have a dataset that consists of tens of thousands of images.

in the dataset, there are some images that have a straight line in their edges, and I want to cut the part that is located outside these straight lines. for example:

the above images contains a horizontal line at the top and a vertical line at the right and some characters outside of the straight line, now I have the code for getting rid of the straight lines (both horizontal and vertical), when I get rid of these lines the image will be like that:

now I got rid of the straight lines, but there is some characters outside the straight line (at right of the straight line) that is messing the data, so I want to get rid of these extra data that are residing outside of the straight lines and crop the image to get rid of them.

that is the code I am using for removing the straight lines:

def remove_lines(img_path, folder_path, img_name_with_extension):
image = cv2.imread(img_path)

# remove horizontal lines in the original image
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV   cv2.THRESH_OTSU)[1]

# Remove horizontal
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25,1))
detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(image, [c], -1, (255,255,255), 2)

# Repair image
repair_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,6))
result = 255 - cv2.morphologyEx(255 - image, cv2.MORPH_CLOSE, repair_kernel, iterations=1)

image = cv2.rotate(image, cv2.ROTATE_90_CLOCKWISE)

# remove horizontal lines in the rotated image
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV   cv2.THRESH_OTSU)[1]

# Remove horizontal
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25,1))
detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    cv2.drawContours(image, [c], -1, (255,255,255), 2)

# Repair image
repair_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,6))
result = 255 - cv2.morphologyEx(255 - image, cv2.MORPH_CLOSE, repair_kernel, iterations=1)

image = cv2.rotate(image, cv2.ROTATE_90_COUNTERCLOCKWISE)

os.chdir(folder_path)
cv2.imwrite(img_name_with_extension, image)

is there any way that I can tweak the code I have written to crop the image and get rid of the straight lines and the characters that are located outside of them?

Thanks.

CodePudding user response：

Perhaps find the centroid of the contour that came from your horizontal line detection--and its boundaries--and use those to crop the image? For instance, after you have your contours from the horizontal line detection portion of your code you could (on a given contour c from your contours cnts) do:

moments = cv2.moments(c)
c_y = int(moments['m01'] / moments['m00'])

And then get the height of the contour:

_, _, _,h = cv2.boundingRect(c)
half_height = int(np.ceil(h/2))

And then crop:

new_image = image[c_y   half_height:, :]

This code assumes the horizontal line is on the top so that you can just default to cropping away everything above it. It seemed that assumption was built into your question. If it is possible for horizontal lines to be at the bottom of the image instead (or as well), you could add a check of whether the centroid of the contour was in the top or the bottom half of the image and then crop away above it or below it, accordingly.

And for the vertical lines, a nearly identical process could be used or even the exact same one if you stick with how your code rotates the image to get the vertical lines using the same method as the horizontal lines.

EDIT:

Actually, you don't even need to compute the moments of the contours. You can do the same process described above simply using the bounding box information:

_, y, _, h = cv2.boundingRect(c)
new_image = image[(y   h   1):, :]

Integrating that last simple solution into some code based off your original code gives a solution like:

import os
import cv2


def detect_hlines_and_crop(image):
    """Remove horizontal lines and anything above them in an image

    """
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    thresh = cv2.threshold(gray,
                           0,
                           255,
                           cv2.THRESH_BINARY_INV   cv2.THRESH_OTSU)
    thresh = thresh[1]
    horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25, 1))
    detected_lines = cv2.morphologyEx(thresh,
                                      cv2.MORPH_OPEN,
                                      horizontal_kernel,
                                      iterations=2)
    cnts = cv2.findContours(detected_lines,
                            cv2.RETR_EXTERNAL,
                            cv2.CHAIN_APPROX_SIMPLE)
    cnts = cnts[0] if len(cnts) == 2 else cnts[1]
    for c in cnts:
        # assumes all lines detected are above the important bit of image
        _, y, _, h = cv2.boundingRect(c)
        image = image[(y   h   1):, :]

    return image


def remove_lines(img_path, folder_path, img_name_with_extension):
    image = cv2.imread(img_path)
    image = detect_hlines_and_crop(image)
    image = cv2.rotate(image, cv2.ROTATE_90_COUNTERCLOCKWISE)
    image = detect_hlines_and_crop(image)
    image = cv2.rotate(image, cv2.ROTATE_90_CLOCKWISE)

    os.chdir(folder_path)
    cv2.imwrite(img_name_with_extension, image)