I am trying to sort boxes which are result of OCR engine. The bounding boxes from the engine are random and not in any particular sorting. I would like to sort the boxes from left to right, top to bottom. Which is the boxes in same row(line) should be sorted left to right, and than it should go to below row(line) and sort it left to right and so on.

I have all the 4 points of each rectangle(box)

Here are some sample Images.