Camera from POCO M3 returns unreadable image bytes-CodePudding

I'm trying to detect faces fith ml_kit_face_detection on flutter. It works fine for other phones. But on POCO M3 ml kit can't detect faces.

I take one of image from camera & save it to file. The file is bellow:

CodePudding user response：

Reverse engineering the format is challenging...

The file format is raw binary format, with some non-standard padding.

I managed to get the following result:

It looks right, but I am not sure that it is entirely correct....

It looks like the file format is NV12 (or NV21) with some padding.
For better understanding NV12 format, you may refer to my following post.

By reverse engineering the file I figured out the following information:

Image resolution: 1280x720
The stored resolution is 1536x720 (there is a padding of 256 pixels in the right side).

File storage format (raw binary format):

Y channel plane: First 1536x719 720 bytes.
UV channel plane: 1536x360 bytes.
Spare bytes, and duplication for UV channel plane.
I don't know why UV channel is duplicated, but the data looks identical.

Illustration:

                     256 padding bytes
                   <----->
        1536 bytes
<------------------------>
YYYYYYYYYYYYYYYYYY00000000
YYYYYYYYYYYYYYYYYY00000000
YYYYYYYYYYYYYYYYYY00000000   719 rows
YYYYYYYYYYYYYYYYYY00000000
YYYYYYYYYYYYYYYYYY00000000
YYYYYYYYYYYYYYYYYY   <--- 1280 bytes (last row of Y)
UVUVUVUVUVUVUVUVUV00000000
UVUVUVUVUVUVUVUVUV00000000  359 rows
UVUVUVUVUVUVUVUVUV00000000
UVUVUVUVUVUVUVUVUV   <---- 1280 bytes (last row of UV)

MATLAB code that reads the data and convert to RGB:

f = fopen('photo', 'r');  % Open binary file for reading.
I1 = fread(f, [1536, 719], '*uint8')'; % Read 1536x719 image (Y channel).
I1last_row = fread(f, 1280, '*uint8')'; % Read 1280 bytes (last row of Y).
I2 = fread(f, [1536, (720-1)/2], '*uint8')'; % Read 1536x359 image (UV channel).
I2last_row = fread(f, 1280, '*uint8')'; % Read 1280 bytes (last row of UV). 
fclose(f);
%figure;imshow(I1);impixelinfo % Show I1 for testing.
%figure;imshow(I2);impixelinfo % Show I2 for testing.

I1 = I1(:, 1:1280); % Crop valid part: 1280x719 pixels.
I1 = [I1; I1last_row]; % Add last row to the bottom (complete to 1280x720).

I2 = I2(:, 1:1280); % Crop valid part (1280x359 UV elements).
I2 = [I2; I2last_row]; % Add last row to the bottom. (complete to 1280x360).

% Convert from NV12 to RGB:
Y = I1;
U = I2(:, 1:2:end); % Extract U color channel 
V = I2(:, 2:2:end); % Extract V color channel 
U = imresize(U, size(Y)); % Resize U to be the same size of Y
V = imresize(V, size(Y)); % Resize V to be the same size of Y
YUV = cat(3, Y, U, V); % Merge Y,U,V channels
RGB = ycbcr2rgb(YUV); % Convert from YCbCr color format to RGB.

% Show and save RGB for testing:
figure;imshow(RGB);impixelinfo 
imwrite('RGB', RGB);

Note:

For better figuring out the format (NV12 or NV21), I suggest you to take a picture with various colors. Post the raw file, and if you can, post a colored reference image for comparison.