subprocess.Popen doesn't redirect output of the spawned process

Question

I am having difficulty redirecting the output of a process created using subprocess.Popen to a file of my choice. The following is what I am doing:

#popen_test.py import sys, os, subprocess, time, signal process_log = open('process.log','w') process = subprocess.Popen([<path_to_python_binary>,"process.py"], stdout=process_log, stderr=process_log, preexec_fn=os.setsid) #launching the sub process time.sleep(10) #waiting for sometime; this allows the sub process to print some statements. See below process_log.flush() os.killpg(process.pid,signal.SIGTERM) process_log.close()

And, this is how my process.py looks. It is basically printing some statements on stdout at regular intervals:

#process.py import sys,os,time print "Initially..." while True: print "Hello World" time.sleep(1)

The process is being launched as I can see using ps, but the output is not going to the process.log--the file remains empty. Any idea what I might be doing wrong?

Try using a full-path to the file when you open it in process_log = open('process.log','w') — Nir Alfasi
– Nir Alfasi, Commented Feb 22, 2015 at 4:19
unlrelated: you could use p.terminate() without setsid hack here because process.py does not create its own child processes. — jfs
– jfs, Commented Feb 22, 2015 at 4:54

Community · Accepted Answer · 2017-05-23 11:58:43Z

It looks like a "block-buffering mode"-related issue. Run the script using python -u or add sys.stdout.flush() after print "hello world".

sys.stdout is fully buffered when redirected to a file by default. print "hello world" adds the string to the stdout buffer. The buffer is not flushed if you kill the subprocess and therefore nothing is written to the file if the buffer does not overflow while the process is still running (buffer size 4K-8K: os.fstat(sys.stdout.fileno()).st_blksize).

Note: the child process works with a copy of the file descriptor i.e., you can close process_log file in the parent as soon as Popen() returns -- it has no effect on the child.

Collectives™ on Stack Overflow

subprocess.Popen doesn't redirect output of the spawned process

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related