Python run piped command and output results live on the screen-CodePudding

I am trying to run a piped command from python3 and would like the contents printed to the screen. After googling for an hour and reading multiple stackoverflow questions I have not been able to execute the command and also output the contents to the screen live.

I am using subprocess but am up to any solution that accomplishes the task. Security is not a requirement. I will need the ability to run multiple commands one by one in order.

import subprocess
from datetime import datetime
import os


currentDateTime = datetime.today().strftime('%Y%m%d_%H%M%S')
domain = "tesla.com"
waybackurlsDir = "/opt/project/recon/{0}/_waybackurls".format(domain)
os.makedirs(waybackurlsDir)
payload = '/usr/bin/echo "{0}" | /root/go-workspace/bin/waybackurls > {1}/{2}_waybackurls.txt'.format(domain, waybackurlsDir, currentDateTime)

process = subprocess.Popen(payload, shell=True, stdout=subprocess.PIPE)
for c in iter(lambda: process.stdout.read(1), b''): 
  sys.stdout.buffer.write(c)
  process.buffer.write(c)

CodePudding user response：

Assuming you want this. Pass parameters to subprocess and capture the output/error of the execution

you can try this

child = subprocess.Popen(['command'], stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE, shell=False)
  out,err=child.communicate('your parameters')
  print(out,err)

CodePudding user response：

I figured out how to solve my problem. Answer is below:

import subprocess
from datetime import datetime
import sys
import os


currentDateTime = datetime.today().strftime('%Y%m%d_%H%M%S')
domain = "tesla.com"
waybackurlsDir = "/opt/myproject/recon/{0}/_waybackurls".format(domain)

if not os.path.exists(waybackurlsDir):
  os.makedirs(waybackurlsDir)

p1 = subprocess.Popen('/usr/bin/echo "tesla.com"', shell=True, stdout=subprocess.PIPE)
p2 = subprocess.Popen('waybackurls', shell=True, stdin=p1.stdout, stdout=subprocess.PIPE)
waybackurlsLogFile = '{0}/{1}_waybackurls.txt'.format(waybackurlsDir, currentDateTime)
waybackurlsFile = open(waybackurlsLogFile, "w ")

while p2.poll() is None:
  for stdout_line in iter(p2.stdout.readline, ""):
    if len(stdout_line) == 0:
      break
    waybackurlsFile.writelines(p2.stdout.readline().decode("utf-8"))
    print("[waybackurls] {0}".format(p2.stdout.readline().rstrip().decode("utf-8")))
  waybackurlsFile.close()