cv2.VideoWriter issues-CodePudding

I'm looking to record a Twitch Livestream by feeding it the direct livestream url using streamlink.streams(url) (which returns a .m3u8 url). With this, I have no problem reading the stream and even writing a few images from it, but when it comes to writing it as a video, I get errors.

P.S.: Yes, I know there's other options like Streamlink and yt-dwl, but I want to operate solely in python, not using CLI... which I believe those two are only dealing with (for recording).

Here's what I currently have:

if streamlink.streams(url):
    stream = streamlink.streams(url)['best']
    stream = str(stream).split(', ')
    stream = stream[1].strip("'")
    cap = cv2.VideoCapture(stream)
    gst_out = "appsrc ! video/x-raw, format=BGR ! queue ! nvvidconv ! omxh264enc ! h264parse ! qtmux ! filesink location=stream "
    out = cv2.VideoWriter(gst_out, cv2.VideoWriter_fourcc(*'mp4v'), 30, (1920, 1080))
    while True:
        _, frame = cap.read()
        out.write(frame)

For this code, I get this error msg:

[tls @ 0x1278a74f0] Error in the pull function.

And if I remove gst_out and feed stream instead as well as moving cap and out into the while loop like so:

if streamlink.streams(url):
    stream = streamlink.streams(url)['best']
    stream = str(stream).split(', ')
    stream = stream[1].strip("'")
    while True:
        cap = cv2.VideoCapture(stream)
        _, frame = cap.read()
        out = cv2.VideoWriter(stream, cv2.VideoWriter_fourcc(*'mp4v'), 30, (1920, 1080))
        out.write(frame)

I get:

OpenCV: FFMPEG: tag 0x7634706d/'mp4v' is not supported with codec id 12 and format 'hls / Apple HTTP Live Streaming'

What am I missing here?

CodePudding user response：

The fist part uses GStreamer syntax, and OpenCV for Python is most likely not built with GStreamer.
The answer is going to be focused on the second part (also because I don't know GStreamer so well).

There are several issues:

cap = cv2.VideoCapture(stream) should be before the while True loop.
out = cv2.VideoWriter(stream, cv2.VideoWriter_fourcc(*'mp4v'), 30, (1920, 1080)) should be before the while True loop.
The first argument of cv2.VideoWriter should be MP4 file name, and not stream.
For getting a valid output file, we have to execute out.release() after the loop, but the loop may never end.

It is recommended to get frame size and rate of the input video, and set VideoWriter accordingly:

 width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
 height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
 fps = int(cap.get(cv2.CAP_PROP_FPS))

 video_file_name = 'output.mp4'

 out = cv2.VideoWriter(video_file_name, cv2.VideoWriter_fourcc(*'mp4v'), fps, (width, height))  # Open video file for writing

It is recommended to break the loop if ret is False:

 ret, frame = cap.read()

 if not ret:
     break

One option to end the recording is when user press Esc key.
Break the loop if cv2.waitKey(1) == 27.
cv2.waitKey(1) is going to work only after executing cv2.imshow.
A simple solution is executing cv2.imshow every 30 frames (for example).

 if (frame_counter % 30 == 0):
     cv2.imshow('frame', frame)  # Show frame every 30 frames (for testing)

 if cv2.waitKey(1) == 27:  # Press Esc for stop recording (cv2.waitKey is going to work only when cv2.imshow is used).
     break

Complete code sample:

from streamlink import Streamlink
import cv2

def stream_to_url(url, quality='best'):
    session = Streamlink()
    streams = session.streams(url)

    if streams:
        return streams[quality].to_url()
    else:
        raise ValueError('Could not locate your stream.')


url = 'https://www.twitch.tv/noraexplorer'  # Need to login to twitch.tv first (using the browser)...
quality='best'

stream_url = stream_to_url(url, quality)  # Get the video URL
cap = cv2.VideoCapture(stream_url, cv2.CAP_FFMPEG)  # Open video stream for capturing

# Get frame size and rate of the input video
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = int(cap.get(cv2.CAP_PROP_FPS))


video_file_name = 'output.mp4'

out = cv2.VideoWriter(video_file_name, cv2.VideoWriter_fourcc(*'mp4v'), fps, (width, height))  # Open video file for writing


frame_counter = 0
while True:
    ret, frame = cap.read()
    
    if not ret:
        break

    if (frame_counter % 30 == 0):
        cv2.imshow('frame', frame)  # Show frame every 30 frames (for testing)

    out.write(frame)  # Write frame to output.mp4

    if cv2.waitKey(1) == 27:  # Press Esc for stop recording (cv2.waitKey is going to work only when cv2.imshow is used).
        break

    frame_counter  = 1

cap.release()
out.release()
cv2.destroyAllWindows()

Testing the setup using FFplay and subprocess module:

from streamlink import Streamlink
import subprocess

def stream_to_url(url, quality='best'):
    session = Streamlink()
    streams = session.streams(url)

    if streams:
        return streams[quality].to_url()
    else:
        raise ValueError('Could not locate your stream.')


#url = 'https://www.twitch.tv/noraexplorer'  # Need to login to twitch.tv first (using the browser)...
url = 'https://www.twitch.tv/valorant'
quality='best'

stream_url = stream_to_url(url, quality)  # Get the video URL

subprocess.run(['ffplay', stream_url])