Open CV imshow() - ARGB32 to OpenCV image-CodePudding

I am trying to process images from Unity3D WebCamTexture graphics format(ARGB32) using OpenCV Python. But I am having trouble interpreting the image on the Open CV side. The image is all Blue (possibly due to ARGB)

try:
    while(True):
        data = sock.recv(480 * 640 * 4)
        if(len(data) == 480 * 640 * 4):
            image = numpy.fromstring(data, numpy.uint8).reshape( 480, 640, 4 )
            #imageNoAlpha = image[:,:,0:2]
            cv2.imshow('Image', image) #further do image processing
            
            key = cv2.waitKey(1) & 0xFF
            if key == ord("q"):
                break
finally:
    sock.close()

CodePudding user response：

The reason is because of the order of the channels. I think the sender read image as a RGB image and you show it as a BGR image or vice versa. Change the order of R and B channels will solve the problem:

image = image[..., [0,3,2,1]] # swap 3 and 1 represent for B and R

You will meet this problem frequently if you work with PIL.Image and OpenCV. The PIL.Image will read the image as RGB and cv2 will read as BGR, that's why all the red points in your image become blue.

CodePudding user response：

OpenCV uses BGR (BGRA when including alpha) ordering when working with color images [1][2], this applies to images read/written with imread(), imwrite(); images acquired with VideoCapture; drawing functions ellipse(), rectangle(); and so on. This convention is self-consistent within the library, if you read an image with imread() and show it with imshow(), the correct colors will appear.

OpenCV is the only library I know that uses this ordering, e.g. PIL and Matplotlib both use RGB. If you want to convert from one color space to another use cvtColor(), example:

# Convert RGB to BGR.
new_image = cvtColor(image, cv2.COLOR_RGB2BGR)

See the ColorConversionCodes enum for all supported conversion pairs. Unfortunately there is no ARGB to BGR, but you can always manually manipulate the NumPy array anyway:

# Reverse channels ARGB to BGRA.
image_bgra = image[..., ::-1]

# Convert ARGB to BGR.
image_bgr = image[..., [3, 2, 1]]

There is also a mixChannels() function and a bunch other array manipulation utilities but most of these are redundant in OpenCV Python since images are backed by NumPy arrays so it's easier to just use the NumPy counterparts instead.

OpenCV uses BGR for seemingly historical reasons: Why OpenCV Using BGR Colour Space Instead of RGB.

References:

[1] OpenCV: Mat - The Basic Image Container (Search for 'BGR' under Storing methods.)

[2] OpenCV: How to scan images, lookup tables and time measurement with OpenCV

Pixel layout in BGR format. ^{Image from [2] showing BGR layout in memory.}

CodePudding user response：

IMAGE_WIDTH = 640
IMAGE_HEIGHT = 480
IMAGE_SIZE = IMAGE_HEIGHT * IMAGE_WIDTH * 4

try:
    while(True):
        data = sock.recv(IMAGE_SIZE)
        dataLen = len(data)
        if(dataLen == IMAGE_SIZE):
            image = numpy.fromstring(data, numpy.uint8).reshape(IMAGE_HEIGHT, IMAGE_WIDTH, 4)
            imageDisp = cv2.cvtColor(image, cv2.COLOR_RGBA2BGR)
            cv2.imshow('Image', imageDisp)
            key = cv2.waitKey(1) & 0xFF
            if key == ord("q"):
                break
finally:
    sock.close()

Edited as per the suggestions from comment