Processing flutter images with C openCV-CodePudding

I am working on a project, where I want to process my images using C OpenCV.

For simplicity's sake, I just want to convert Uint8List to cv::Mat and back.

Following this tutorial, I managed to make a pipeline that doesn't crash the app. Specifically:

I created a function in a .cpp that takes the pointer to my Uint8List, rawBytes, and encodes it as a .jpg:

    int encodeIm(int h, int w, uchar *rawBytes, uchar **encodedOutput) {
        cv::Mat img = cv::Mat(h, w, CV_8UC3, rawBytes); //CV_8UC3
        vector<uchar> buf;
        cv:imencode(".jpg", img, buf); // save output into buf. Note that Dart Image.memory can process either .png or .jpg, which is why we're doing this encoding
        *encodedOutput = (unsigned char *) malloc(buf.size());
        for (int i=0; i < buf.size(); i  )
            (*encodedOutput)[i] = buf[i];
        return (int) buf.size();
    }

Then I wrote a function in a .dart that calls my c encodeIm(int h, int w, uchar *rawBytes, uchar **encodedOutput):

   //allocate memory heap for the image
   Pointer<Uint8> imgPtr = malloc.allocate(imgBytes.lengthInBytes);

   //allocate just 8 bytes to store a pointer that will be malloced in C   that points to our variably sized encoded image
   Pointer<Pointer<Uint8>> encodedImgPtr = malloc.allocate(8);

   //copy the image data into the memory heap we just allocated
   imgPtr.asTypedList(imgBytes.length).setAll(0, imgBytes);

   //c   image processing
   //image in memory heap -> processing... -> processed image in memory heap
   int encodedImgLen = _encodeIm(height, width, imgPtr, encodedImgPtr);
   //

   //retrieve the image data from the memory heap
   Pointer<Uint8> cppPointer = encodedImgPtr.elementAt(0).value;
   Uint8List encodedImBytes = cppPointer.asTypedList(encodedImgLen);
   //myImg = Image.memory(encodedImBytes);
   return encodedImBytes;
   //free memory heap
   //malloc.free(imgPtr);
   //malloc.free(cppPointer);
   //malloc.free(encodedImgPtr); // always frees 8 bytes
 }

Then I linked c with dart via:

final DynamicLibrary nativeLib = Platform.isAndroid
    ? DynamicLibrary.open("libnative_opencv.so")
    : DynamicLibrary.process();

final int Function(int height, int width, Pointer<Uint8> bytes, Pointer<Pointer<Uint8>> encodedOutput) 
_encodeIm = nativeLib
        .lookup<NativeFunction<Int32 Function(Int32 height, Int32 width, 
Pointer<Uint8> bytes, Pointer<Pointer<Uint8>> encodedOutput)>>('encodeIm').asFunction();

And finally I show the result in Flutter via:

Image.memory(...)

Now, the pipeline doesn't crash, which means I haven't goofed up memory handling completely, but it doesn't return the original image either, which means I did mess up somewhere.

Original image:

Pipeline output:

CodePudding user response：

Thanks to Richard Heap's guidance in the comments, I managed to fix the pipeline by changing my matrix definition from

cv::Mat img = cv::Mat(h, w, CV_8UC3, rawBytes);

vector<uint8_t> buffer(rawBytes, rawBytes   inBytesCount);
Mat img = imdecode(buffer, IMREAD_COLOR);

where inBytesCount is the length of imgBytes.

If you found this useful, please, upvote both the question and the answer, because I use karma to pay for bounties.