python 3: in process substitution, child fails to open file descriptor [duplicate]-CodePudding

I've created a MCVE to show the problem:

test-process-subst.py:

from subprocess import Popen
import sys
Popen(['/home/slu/util/argopen']   sys.argv[1:])

argopen:

#!/bin/bash
 
ITERATION=1
while (( "$#" )); do 
  echo '$'"${ITERATION}: ${1}"
  echo contents: "$(cat "$1")"
  shift
  (( ITERATION    ))
done

Inside zsh shell:

❯ echo 1 > one
❯ python ../test-process-subst.py one <(echo 2)
$1: one
contents: 1
$2: /proc/self/fd/11
contents: 2
❯ python3 ../test-process-subst.py one <(echo 2)   
$1: one                                            
contents: 1                                        
$2: /proc/self/fd/11                               
cat: /proc/self/fd/11: No such file or directory   
contents:

The problem seems to be something related to the execution context of the subprocess under python 3. Unlike with Python 2, child is not able to open the parent's file descriptor.

CodePudding user response：

Popen(..., close_fds=False) brings back python2 behavior for python3.

CodePudding user response：

If i've understood this https://www.python.org/dev/peps/pep-0446/#abstract correctly, it's a feature introduced in python 3.4, to reduce the securety issues of inherited file descriptors.

As soon as you open a process with Popen, /proc/self/fd/11 is closed from the child point of view, unless Popen(..., close_fds=False) is specified.

To see this, just print(os.listdir('/proc/self/fd')) right before the Popen call