How to convert output from subprocess to list of dictionaries-CodePudding

For one of my task to call a python script and get the output from the script and do some operations on that. I have written a below code.

import subprocess
import json
chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command= """python predict_icd.py --cc """ str(chief_complaint)
command_proc=subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
print("response - ",command_proc.stdout.read())

The output that I am getting of above code is

command_proc.stdout.read() -  b"[{'text': 'CHA', 'start': 51, 'end': 54, 'code': ''}, {'text': 'CHA', 'start': 221, 'end': 224, 'code': ''}, {'text': 'dog', 'start': 286, 'end': 289, 'code': ''}, {'text': 'Schizophrenia', 'start': 351, 'end': 364, 'code': 'F20'}, {'text': 'chronic pain', 'start': 366, 'end': 378, 'code': 'G89.29'}, {'text': 'chest pain', 'start': 599, 'end': 609, 'code': 'R07.89'}, {'text': 'palpitations', 'start': 613, 'end': 625, 'code': 'R00.2'}, {'text': 'cough', 'start': 829, 'end': 834, 'code': 'R05'}, {'text': 'wheezing', 'start': 836, 'end': 844, 'code': 'R06.2'}, {'text': 'pain', 'start': 966, 'end': 970, 'code': ''}, {'text': 'schizophrenia', 'start': 1078, 'end': 1091, 'code': 'F20'}]\n"

I want to convert this output into list of dictionaries to perform some action. if there is any alternative way to call the script and get the output please suggest the better way.

CodePudding user response：

The absolutely best solution is to not run Python as a subprocess of itself. Instead, refactor predict_icd.py so you can import it and call its functions directly. See What is the best way to call a script from another script?

Your second best option is to change predict_icd.py so that it prints its results in a standard format like JSON. Then you can call it from any tool, not just Python.

The distant third option is to go with what you have and use ast.literal_eval() to parse out the results. I have also refactored to avoid the pesky and unnecessary shell=True and fix the orphaned Popen.

import subprocess
# import json  # unused import
from ast import literal_eval

chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command_proc = subprocess.run(
    ['python', 'predict_icd.py', '--cc', chief_complaint],
    capture_output=True, text=True, check=True)
response = literal_eval(command_proc.stdout)
print("response = ", response)

In some more detail, Popen will only start a process; you have to take care of it during its lifetime if you do that - but subprocess.run() already takes care of all this plumbing for you. Like its documentation says, avoid Popen when the higher-level functions from the standard library already do what you want.

Notice also the text=True which tells Python to send and receive text, not bytes to and from the subprocess.