Asynchronously read stdout from subprocess.Popen

Question

I am running a sub-program using subprocess.popen. When I start my Python program from the command window (cmd.exe), the program writes some info and dates in the window as the program evolves.

When I run my Python code not in a command window, it opens a new command window for this sub-program's output, and I want to avoid that. When I used the following code, it doesn't show the cmd window, but it also doesn't print the status:

p = subprocess.Popen("c:/flow/flow.exe", shell=True, stdout=subprocess.PIPE) print p.stdout.read()

How can I show the sub-program's output in my program's output as it occurs?

"windows window" What Windows window? Are you using a GUI framework? Which one? — Mark Byers
– Mark Byers, Commented Nov 30, 2009 at 20:38
well, i am running the model through arcgis. when i click my tool i created over there, a window comes through and shows the progress. I want to see lines appearing in my command window. — Mesut
– Mesut, Commented Nov 30, 2009 at 21:43
python3 w/ asyncio this is what you want: kevinmccarthy.org/2016/07/25/… — JeffCharter
– JeffCharter, Commented Apr 26, 2021 at 19:24

Roger Pate · Accepted Answer · 2010-02-22 11:08:14Z

Use this:

cmd = subprocess.Popen(["c:/flow/flow.exe"], stdout=subprocess.PIPE) for line in cmd.stdout: print line.rstrip("\n") cmd.wait() # you may already be handling this in your current code

Note that you will still have to wait for the sub-program to flush its stdout buffer (which is commonly buffered differently when not writing to a terminal window), so you may not see each line instantaneously as the sub-program prints it (this depends on various OS details and details of the sub-program).

Also notice how I've removed the shell=True and replaced the string argument with a list, which is generally recommended.

alternatively to the 'line.rstrip..' you can use 'line,' (note the trailing comma)
for line in iter(cmd.stdout.readline, ""): print line, might provide a more immediate output by possibly avoiding read-ahead buffer in the file iterator (note: the sub-program still have to flush its stdout)

Tino · Accepted Answer · 2011-10-30 17:18:58Z

Looking for a recipe to process Popen data asynchronously I stumbled upon http://code.activestate.com/recipes/576759-subprocess-with-async-io-pipes-class/

This looks quite promising, however I got the impression that there might be some typos in it. Not tried it yet.

fviktor · Accepted Answer · 2011-04-08 20:46:38Z

It is an old post, but a common problem with a hard to find solution. Try this: http://code.activestate.com/recipes/440554-module-to-allow-asynchronous-subprocess-use-on-win/

Collectives™ on Stack Overflow

Asynchronously read stdout from subprocess.Popen

3 Answers 3

3 Comments

Comments

1 Comment

Linked

Hot Network Questions