Read video from base64 in python-CodePudding

I would like to process a video that I receive through an upload component in plotly dash. Currently I am creating a tempfile and then reading the video with opencv. However, I would like to avoid writing the file to disk. Is there a way to process the video directly from memory? My code looks like this:

def process_motion(contents, filename):
    print("Processing video: "   filename)
    content_type, content_string = contents.split(',')

    decoded = base64.b64decode(content_string)

    with tempfile.NamedTemporaryFile() as temp:
        temp.write(decoded)

        cap = cv2.VideoCapture(temp.name)

        frames = cap.get(cv2.CAP_PROP_FRAME_COUNT)
        fps = cap.get(cv2.CAP_PROP_FPS)
        
        # some calculations

CodePudding user response：

Interesting question

as @Dan Mašek has correctly mentioned, the correct keyword to search would be video decoding. The thing is, like OpenCV, must of decoding libraries in python using FFMPEG as the backend and most of them are wrappers around the FFMPEG executable (run FFMPEG in subprocess), and since FFMPEG accepts file path for decoding (or URL for decoding video stream) those wrappers also accept file path.

AFAIK, the only wrapper library which accepts bytes as input to decode video and get all frames is imageio which behind the scene also converts bytes to a temp file and decode it using FFMPEG.

Here is the comparison of using imageio and your solution.

import base64
import imageio.v3 as iio
import cv2
from time import perf_counter
import tempfile
import numpy as np

with open("example.mp4", "rb") as videoFile:
    base_text = base64.b64encode(videoFile.read())


buffer = base64.b64decode(base_text)

start = perf_counter()
frames = iio.imread(buffer, index=None, format_hint=".mp4")
print("iio ffmpeg pluing: ", perf_counter() - start)
print(frames.shape)


start = perf_counter()
frames = iio.imread(buffer, index=None, plugin="pyav", format_hint=".mp4")
print("iio pyav pluing: ", perf_counter() - start)
print(frames.shape)

start = perf_counter()
with tempfile.NamedTemporaryFile() as temp:
    temp.write(buffer)

    cap = cv2.VideoCapture(temp.name)

    frames = cap.get(cv2.CAP_PROP_FRAME_COUNT)
    fps = cap.get(cv2.CAP_PROP_FPS)
    success, frame = cap.read()
    frames = [frame]
    while success:
        success, frame = cap.read()
        frames.append(frame)
    frames = np.stack(frames[:-1])
print("cv2: ", perf_counter() - start)
print(frames.shape)
============================================
iio ffmpeg pluing:  0.3905044999992242
(901, 270, 480, 3)
No accelerated colorspace conversion found from yuv420p to rgb24.
iio pyav pluing:  0.3710011249931995
(901, 270, 480, 3)
cv2:  0.2388888749992475
(901, 270, 480, 3)

Note: if you are able to create a python wrapper for FFMPEG it is possible to convert a bytes array to a format that is acceptable FFMPEG's stream decoding function. as explained here