the task I want to do looks pretty simple: I take as input several images with an object centered in the photo and a little color chart needed for other purposes. My code normally works for the majority of the cases, but sometimes fails miserably and I just can't understand why.

For example (these are the source images), it works correctly on this

Output images:

The green outlines are the contours of the objects, and the red outlines are the bounding boxes of the objects.