Home > Software engineering >  Python CSV module returning non string iterator when passed the STDOUT stream containing CSV data
Python CSV module returning non string iterator when passed the STDOUT stream containing CSV data

Time:12-13

The following program calls subprocess.run() to invoke a program (ffprobe, but that's not important) and return a CSV result in STDOUT. I am getting an error enumerating the CSV result.

import os
import subprocess
import csv

for file in os.listdir("/Temp/Video"):
    if file.endswith(".mkv"):
        print(os.path.join("/Temp/Video", file))
        ps = subprocess.run(["ffprobe", "-show_streams", "-print_format", "csv",  "-i", "/Temp/Video/"   file], capture_output = True)
        print(ps.stdout)
        reader = csv.reader(ps.stdout)
        results = []
        for row in reader:
            results.append(row)

Produces the following error:

Traceback (most recent call last):
  File "C:\Users\mbmas\eclipse-workspace\DolbyAtmosToFlac\com\mbm\main.py", line 12, in <module>
    for row in reader:
_csv.Error: iterator should return strings, not int (the file should be opened in text mode)

The output from the print(ps.stdout) statement produces:

b'stream,0,h264,H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10,High,video,[0][0][0][0],0x0000,1920,1080,1920,1080,0,0,2,1:1,16:9,yuv420p,40,unknown,unknown,unknown,unknown,left,progressive,1,true,4,N/A,24000/1001,24000/1001,1/1000,0,0.000000,N/A,N/A,N/A,N/A,8,N/A,N/A,N/A,46,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,eng,17936509,01:20:18.271791666,115523,10802870592,001011,MakeMKV v1.16.4 win(x64-release),2021-08-20 19:09:26,BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID,Lavc59.7.102 libx264,00:01:30.010000000\r\nstream,1,vorbis,Vorbis,unknown,audio,[0][0][0][0],0x0000,fltp,48000,3,3.0,0,0,N/A,0/0,0/0,1/1000,0,0.000000,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,3314,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,eng,Surround 3.0,2422660,01:20:18.272000000,451713,1459129736,001100,MakeMKV v1.16.4 win(x64-release),2021-08-20 19:09:26,BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES SOURCE_ID,Lavc59.7.102 libvorbis,00:01:30.003000000\r\n'

The suggestion Python provides, the file should be opened in text mode, is confusing. The above output looks like text to me. How do I use the csv module to read CSV data held in a stream?

CodePudding user response:

subprocess.run is returning bytes:

b'stream,0,h264,H.264 ,,,'

To have it return text, pass text=True:

ps = subprocess.run(
    ["ffprobe", "-show_streams", "-print_format", "csv",  "-i", "/Temp/Video/"   file],
    capture_output=True,
    text=True
)
  • Related