Qt: getting a thumbnail for a video-CodePudding

Is there a better way to get a thumbnail for a video? OS is primarily Linux, but hopefully there's a cross platform way to do it. This is what I have right now:

from PySide6 import QtMultimedia as qtm
from PySide6 import QtMultimediaWidgets as qtmw
from PySide6 import QtCore as qtc

app = qtw.QApplication()

thumbnail_file = "video.mp4"
loop = qtc.QEventLoop()
widget = qtmw.QVideoWidget()
widget.setVisible(False)
media_player = qtm.QMediaPlayer()
media_player.setVideoOutput(widget)
media_player.mediaStatusChanged.connect(loop.exit)
media_player.positionChanged.connect(loop.exit)
media_player.setSource(thumbnail_file)
loop.exec()
media_player.mediaStatusChanged.disconnect()
media_player.play()
if media_player.isSeekable():
    media_player.setPosition(media_player.duration() // 2)
loop.exec()
media_player.positionChanged.disconnect()
media_player.stop()
image = media_player.videoSink().videoFrame().toImage()
image.save('thumbnail.jpg')
app.exec()

This will be ran in a separate thread so the time is not really an issue, but it's still pretty convoluted.

CodePudding user response：

Answer reached thanks to the discussion in the comments of OP:

from PySide6 import QtWidgets as qtw
from PySide6 import QtMultimedia as qtm
from PySide6 import QtMultimediaWidgets as qtmw
from PySide6 import QtCore as qtc

app = qtw.QApplication()

thumbnail_file = "video.mp4"
loop = qtc.QEventLoop()
video_sink = qtm.QVideoSink()
media_player = qtm.QMediaPlayer()
media_player.setVideoSink(video_sink)
media_player.mediaStatusChanged.connect(loop.exit)
media_player.setSource(thumbnail_file)
loop.exec()
media_player.mediaStatusChanged.disconnect()
if media_player.isSeekable() and media_player.duration():
    media_player.positionChanged.connect(loop.exit)
    media_player.setPosition(media_player.duration() // 2)
    media_player.play()
    loop.exec()
    media_player.positionChanged.disconnect()
else:
    media_player.play()
video_sink.videoFrameChanged.connect(loop.exit)
loop.exec()
video_sink.videoFrameChanged.disconnect()
media_player.stop()
image = media_player.videoSink().videoFrame().toImage()
image.save('thumbnail.jpg')

CodePudding user response：

There are many different approaches to this, since QMediaPlayer can undergo multiple state changes which can be monitored in a variety of ways. So the question of what is "best" probably comes down to what is most reliable/predictable on the target platform(s). I have only tested on Arch Linux using Qt-6.3.1 with a GStreamer-1.20.3 backend, but I think the solution presented below should work correctly with most other setups.

As it stands, the example code in the question doesn't work on Linux. As pointed out in the comments, this can be fixed by using the videoFrameChanged signal of the mediad-player's video-sink. (Unfortunately, at time of writing, these APIs are rather poorly documented). However, the example can be simplified and improved in various ways, so I have provided an alternative solution below.

During testing, I found that the durationChanged and postionChanged signals cannot be relied upon to give the relevant notifications at the appropriate time, so I have avoided using them. It's best to wait for the player to reach the buffered-state before setting the position, and then wait until a frame is received that can be verified as matching the requested position before saving the image. (I would also advise adding a suitable timeout in case the media-player gets stuck in an indeterminate state).

To illustrate the timing issues alluded to above, here is some sample output:

frame changed: 0
duration change: 13504
status change: b'LoadedMedia'
frame changed: 0
status change: b'BufferedMedia'
set position: 6752
frame changed: 6752
save: exit

frame changed: 0
frame changed: 0
duration change: 13504
status changed: b'LoadedMedia'
status changed: b'BufferedMedia'
set position: 6752
frame changed: 6752
save: exit

As you can see, the exact sequence of events isn't totally predictable. If the position is set before the buffered-state is reached, the output looks like this:

frame changed: 0
duration changed: 13504
status changed: LoadedMedia
set position: 0
frame changed: 0
status change: BufferedMedia
frame changed: 40
frame changed: 80
... # long list of changes
frame changed: 6720
frame changed: 6760
save: exit

Here is the demo script (which works with both PySide6 and PyQt6):

import os
from PySide6.QtCore import QCoreApplication, QTimer, QEventLoop, QUrl
from PySide6.QtMultimedia import QMediaPlayer, QVideoSink
# from PyQt6.QtCore import QCoreApplication, QTimer, QEventLoop, QUrl
# from PyQt6.QtMultimedia import QMediaPlayer, QVideoSink

def thumbnail(url):
    position = 0
    image = None
    loop = QEventLoop()
    QTimer.singleShot(15000, lambda: loop.exit(1))
    player = QMediaPlayer()
    player.setVideoSink(sink := QVideoSink())
    player.setSource(url)
    def handle_status(status):
        nonlocal position
        print('status changed:', status.name)
        # if status == QMediaPlayer.MediaStatus.LoadedMedia:
        if status == QMediaPlayer.MediaStatus.BufferedMedia:
            player.setPosition(position := player.duration() // 2)
            print('set position:', player.position())
    def handle_frame(frame):
        nonlocal image
        print('frame changed:', frame.startTime() // 1000)
        if (start := frame.startTime() // 1000) and start >= position:
            sink.videoFrameChanged.disconnect()
            image = frame.toImage()
            print('save: exit')
            loop.exit()
    player.mediaStatusChanged.connect(handle_status)
    sink.videoFrameChanged.connect(handle_frame)
    player.durationChanged.connect(
        lambda value: print('duration changed:', value))
    player.play()
    if loop.exec() == 1:
        print('ERROR: process timed out')
    return image

video_file = 'video.mp4'
thumbnail_file = 'thumbnail.jpg'

try:
    os.remove(thumbnail_file)
except OSError:
    pass

app = QCoreApplication(['Test'])

image = thumbnail(QUrl.fromLocalFile(video_file))
if image is not None:
    image.save(thumbnail_file)