I am trying to display received WebRTC frames using OpenCV imshow(). WebRTC delivers frames as objects of webrtc::VideoFrame and in my case, I can access webrtc::I420Buffer from it. Now my question is how do I convert the data in webrtc::I420Buffer to cv::Mat, so that I can give it to imshow()?

Thsi is what the definition of webrtc::I420Buffer looks like

namespace webrtc {

// Plain I420 buffer in standard memory.
class RTC_EXPORT I420Buffer : public I420BufferInterface {
 public:
  ...

  int width() const override;
  int height() const override;
  const uint8_t* DataY() const override;
  const uint8_t* DataU() const override;
  const uint8_t* DataV() const override;

  int StrideY() const override;
  int StrideU() const override;
  int StrideV() const override;

  uint8_t* MutableDataY();
  uint8_t* MutableDataU();
  uint8_t* MutableDataV();

  ...

 private:
  const int width_;
  const int height_;
  const int stride_y_;
  const int stride_u_;
  const int stride_v_;
  const std::unique_ptr<uint8_t, AlignedFreeDeleter> data_;
};

CodePudding user response：

The main issue is converting from I420 color format to BGR (or BGRA) color format used by OpenCV.

Two good options for color conversion:

• Using sws_scale - part of the C interface libraries of

  ---

  Note:
  It seems to difficult to actually use WebRTC input stream.
  The assumption is that the decoded raw video frame already exists in I420Buffer as defined in your post.