I am trying to transmit RGB raw data over the network via UDP and on the receiving side, I want to do some image processing on the data using OpenCV C .

Sender Pipeline

gst-launch-1.0 -v k4asrc serial=$k4a_serial timestamp-mode=clock_all enable-color=true ! rgbddemux name=demux demux.src_color ! rtpvrawpay ! udpsink host=192.168.100.46 port=9001

Receiver:

#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
#include <iostream>

using namespace std;
using namespace cv;

int main()
{
    VideoCapture cap("udpsrc port=9001 ! application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=RAW, sampling=(string)RGB, width=(string)1280, height=(string)720, payload=96 ! rtpvrawdepay ! videoconvert ! queue ! appsink", CAP_GSTREAMER);

    if (!cap.isOpened()) {
        cerr << endl <<"VideoCapture not opened !!!" << endl;
        exit(-1);
    }

    while (true) {

        Mat frame;

        cap.read(frame);

        imshow("receiver", frame);

        if (waitKey(1) == 27) {
            break;
        }
    }

    return 0;
}

I get the following output in the terminal:

[ INFO:[email protected]] global C:\build\master_winpack-build-win64-vc15\opencv\modules\videoio\src\videoio_registry.cpp (223) cv::`anonymous-namespace'::VideoBackendRegistry::VideoBackendRegistry VIDEOIO: Enabled backends(8, sorted by priority): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); DSHOW(960); CV_IMAGES(950); CV_MJPEG(940); UEYE(930)
[ INFO:[email protected]] global C:\build\master_winpack-build-win64-vc15\opencv\modules\videoio\src\backend_plugin.cpp (383) cv::impl::getPluginCandidates Found 2 plugin(s) for GSTREAMER
[ INFO:[email protected]] global c:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\utils\plugin_loader.impl.hpp (67) cv::plugin::impl::DynamicLib::libraryLoad load C:\opencv\build\x64\vc15\bin\opencv_videoio_gstreamer455_64d.dll => FAILED
[ INFO:[email protected]] global c:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\utils\plugin_loader.impl.hpp (67) cv::plugin::impl::DynamicLib::libraryLoad load opencv_videoio_gstreamer455_64d.dll => FAILED

VideoCapture not opened !!!

C:\Users\username\g-streamer\gst_sample\gst-udp-03\out\build\x64-Debug\gstreamer_udp_03.exe (process 21264) exited with code -1.
Press any key to close this window . . .

Why does the opencv_videoio_gstreamer455_64d.dll load fail? I checked and it is already in the provided path.

When I try the same Pipeline in the terminal, I can see the expected video.

Terminal Receiver Pipeline:

gst-launch-1.0 udpsrc port=9001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)RGB, width=(string)1280, height=(string)720, payload=(int)96" ! rtpvrawdepay ! videoconvert ! queue ! autovideosink sync=false

System:

Sender-PC = Ubuntu 18.04
Receiver-PC = Windows 10
Python = 3.7.9
OpenCV = 4.5.5 

CodePudding user response：

SOLVED after several tries. Hope it helps someone else.

I reinstalled everything. Built another version of OpenCV and it's working as expected. Also had to change the Pipeline a bit to add all the CAPS information.

Sender Pipeline

gst-launch-1.0 -v k4asrc serial=$k4a_serial timestamp-mode=clock_all enable-color=true ! rgbddemux name=demux demux.src_color ! rtpvrawpay ! udpsink host=192.168.100.46 port=9001

Receiver:

#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
#include <iostream>

using namespace std;
using namespace cv;

int main()
{
    // Receiving Pipeline
    VideoCapture cap("udpsrc port=9001 ! application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)RGB, depth=(string)8, width=(string)1280, height=(string)720, colorimetry=(string)SMPTE240M, payload=(int)96, ssrc=(uint)597927397, timestamp-offset=(uint)407659256, seqnum-offset=(uint)20318, a-framerate=(string)30 ! rtpvrawdepay ! videoconvert ! appsink", CAP_GSTREAMER);


    if (!cap.isOpened()) {
        cerr << endl <<"VideoCapture not opened !!!" << endl;
        exit(-1);
    }

    while (true) {

        Mat frame;

        cap.read(frame);

        imshow("receiver", frame);

        if (waitKey(1) == 27) {
            break;
        }
    }
    // When everything done, release the video capture object

    cap0.release();
    cap1.release();

    // Closes all the frames
    destroyAllWindows();

    return 0;
}

System:

Sender-PC = Ubuntu 18.04
Receiver-PC = Windows 10
OpenCV = 4.2.0
Gstreamer = 1.20.1
Python = 3.8.9