subprocess.Popen() results taking a very long time to communicate() in Linux-CodePudding

I'm working on a project where I am attempting to scan for any application communicating through TCP port 587. In windows, I was able to use the code below:

import subprocess
from datetime import datetime

proc = subprocess.Popen('netstat -ano -p tcp | findStr "587"', shell=True, stdin=subprocess.PIPE,
                        stdout=subprocess.PIPE)
print(f'before proc.communicate(): {datetime.now()}')
out, err = proc.communicate()
print(f'after: {datetime.now()})

With these results:

before proc.communicate(): 2022-03-12 17:21:04.079038
after: 2022-03-12 17:21:12.411754

However, when I try to run an alternative command in Linux below:

import subprocess
from datetime import datetime

proc = subprocess.Popen('lsof -nP -iTCP:587',
                        shell=True,
                        stdout=subprocess.PIPE)
print(f'before proc.communicate(): {datetime.now()}')
# this line is taking way too long to run
(out, err) = proc.communicate()
print(f'after: {datetime.now()}')

My results are as follows:

before proc.communicate(): 2022-03-12 14:31:23.652261
after: 2022-03-12 14:31:36.613435

I'm not sure using datetime is giving the most accurate results... When I run the command in Linux from my IDE (PyCharm), it takes several seconds to show the "after..." results while on Windows it is instant. Any help to work this issue out or better understand what's going on is very much appreciated! Thanks.

CodePudding user response：

Give psutil a try. This might solve slowness and provide python objects instead of raw text.

import psutil

psutil.net_connections(kind='tcp')

CodePudding user response：

Just use netstat, and let Python do the text filtering.

I don't have a Linux box handy; this is the output on FreeBSD 13.

In [1]: import subprocess as sp

In [2]: %timeit sp.run(["netstat", "-an", "-p", "tcp"], capture_output=True, text=True)
3.04 ms ± 20 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [3]: r = sp.run(["netstat", "-an", "-p", "tcp"], capture_output=True, text=True);

In [4]: %timeit [ln for ln in r.stdout.splitlines() if ".587 " in ln]
6.81 µs ± 7.74 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [5]: port587 = [ln for ln in r.stdout.splitlines() if ".587 " in ln]
Out[5]: []

Note how running the external process takes a couple of milliseconds, but filtering the output only takes microseconds.