Responding to interactive programs in bash with python subprocess-CodePudding

I have the bash script like the one below below. When you run it, it asks you for your name and then greets you:

#example.sh
printf "Hello, who is running this program? "

read -r USERNAME

echo Hi $USERNAME!

Is it possible to write a python script that can 1. Run sh example.sh 2. Respond python to the prompt for a name, and 3. Read Hi Larry! back into python?

And just to clarify this is a hypothetical situation, so modifying the bash script is not the goal. Thank you!

CodePudding user response：

If you know all answers for external script then you can use subprocess.run() and send all answers (with \n) as one string.

For test I use script which wait for two values USERNAME and COLOR

#example.sh
printf "Hello, who is running this program? "

read -r USERNAME

echo Hi $USERNAME!

printf "What is your favorite color? "

read -r COLOR

echo I like $COLOR too!

And I can use single string "Larry\nBlue\n" to send both values at once.

import subprocess

cmd = ['sh', 'example.sh']

p = subprocess.run(cmd, input="Larry\nBlue\n".encode(), stdout=subprocess.PIPE)

print( p.stdout.decode() )

It sends all at start and external script get first line as first answer, second line as second answer, etc. And when it finish then Python can get all output -

Hello, who is running this program? Hi Larry!
What is your favorite color? I like Blue too!

and you may have to remove elements which you don't need.

But if need to get output from first answer from external script to decide what to send as next answer then it makes problem.

It would need subprocess.Popen() to run and at the same time send input and read output. But there is other problem. If external script send line with \n then you can read it with readline() but if it doesn't send \n then readline() will wait for \n and it will block Python code. The same will be if it send one line but you will try to read 2 lines. Second readline() will block it. So you have to know how many lines it sends.

If you use read() then it will wait for end of data and it will also block Python. If you use read(100) and output will have 100 bytes (or more) then it will read 100 bytes (and rest will wait for next read() - but if it will have 99 bytes then read(100) will block Python.

You will have to know how may bytes to read or you have to know what text it may send and then you may read(1), add it to buffer, and check if buffer has string which you expect. And this method uses expect and pyexpect.

First example with readline()

I use example.sh with \n in printf

printf "Hello, who is running this program? \n"

read -r USERNAME

echo Hi $USERNAME!

printf "What is your favorite color? \n"

read -r COLOR

echo I like $COLOR too!

Because example.sh send with \n so I remove \n

import subprocess

cmd = ['sh', 'example.sh']

p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    
print('1 >>>', p.stdout.readline().decode().rstrip(), '<<<')
p.stdin.write("Larry!\n".encode())
p.stdin.flush()
print('2 >>>', p.stdout.readline().decode().rstrip(), '<<<')

print('1 >>>', p.stdout.readline().decode().rstrip(), '<<<')
p.stdin.write("Blue!\n".encode())
p.stdin.flush()
print('2 >>>', p.stdout.readline().decode().rstrip(), '<<<')

Result:

1 >>> Hello, who is running this program? <<<
2 >>> Hi Larry!! <<<
1 >>> What is your favorite color? <<<
2 >>> I like Blue! too! <<<

EDIT:

Example in pexpect

Similar to first version - I send all answers and get all output

import pexpect    

p = pexpect.spawn('sh script.sh')

p.sendline('Larry')
p.sendline('Blue')

p.expect(pexpect.EOF)  # wait for output

print(p.before.decode())

Similar to second example. Pexpect send input as output so it need extra `readline()

import pexpect    

p = pexpect.spawn('sh script.sh')

# ---
    
p.readline()
print('output 1 >>>', p.before.decode(), '<<<')

p.sendline('Larry')

p.readline()
print('input    >>>', p.before.decode(), '<<<')
p.readline()
print('output 2 >>>', p.before.decode(), '<<<')

# ---
    
p.readline()
print('output 1 >>>', p.before.decode(), '<<<')

p.sendline('Blue')

p.readline()
print('input    >>>', p.before.decode(), '<<<')
p.readline()
print('output 2 >>>', p.before.decode(), '<<<')

p.expect(pexpect.EOF)  # get rest to the end
print('end:', p.before.decode())

Instead of readline() it can use expect(regex) or expect_exact(text)

import pexpect    

p = pexpect.spawn('sh script.sh')

# ---
    
p.expect_exact('\r\n')
print('output 1 >>>', p.before.decode(), '<<<')

p.sendline('Larry')

p.expect_exact('\r\n')
print('input    >>>', p.before.decode(), '<<<')
p.expect_exact('\r\n')
print('output 2 >>>', p.before.decode(), '<<<')

# ---
    
p.expect_exact('\r\n')
print('output 1 >>>', p.before.decode(), '<<<')

p.sendline('Blue')

p.expect_exact('\r\n')
print('input    >>>', p.before.decode(), '<<<')
p.expect_exact('\r\n')
print('output 2 >>>', p.before.decode(), '<<<')

p.expect(pexpect.EOF)  # get rest to the end
print('end:', p.before.decode())