How could I find simple rectangles in image such as:

save them in separate picture files and replace them by a simple description such as:

CodePudding user response：

I assume this is an RGB image.
One possible solution would be to use OpenCV or Pillow to convert the image into an array (3 channels). Then set a threshold for the colours.
For example, the red colour is RBG=(255,0,0). You could set a threshold of more than 200 for the first channel and less than 5 for the second and third channels. Then you determine the coordinates of the localised area and get the resulting array.
Then, you convert this array to white. So, replace this array with a new array (having the same dimension) with the white colour RGB=rgb(255,255,255).
Since you already know the position of the localised rectangle, you can label the new image with PIL, for example.

CodePudding user response：

Turn the image to binary by assigning white where you see the color of a rectangle and black elsewhere.

Detect the connected components and determine their bounding box.

Now you can replace the content of the image in these bounding boxes.

Repeat for every rectangle color.

There must be a rule to tell which text to write in which box. But you did not state that.

CodePudding user response：

I suggest you look up text recognition and neural networks.

1. You can use various methods to find the rectangles and get the coordinates of the corners.
2. Then you will need to use a neural network or a module that can recognize the characters at those coordinates and return them as a string in your program.
3. Then use a translation module to translate that text. Maybe try the GoogleTrans module.
4. Lastly, use any sort of GUI or image module to insert a text box at those coordinates with the translated text inside.

I have a link for translation, and I will try to find some modules that can do the other steps.