0

I have a listener that says "waiting for messages..." and then runs forever waiting for messages from all sorts of programs to pass to a responder. All of this works fine when I run it from the command line, but now I want to automate it so I don't have to start all my programs separately.

I started out by just trying to get my listener to start, but it doesn't print out the "waiting for messages..." message, or anything at all, when I try to route stdout to a file. 

from subprocess import Popen, PIPE with open('listenerOutput', 'w') as outfile: Popen(["python", "listener.py"], stdout = outfile)

All the questions I've read so far seem to be more complex than this, but I can't even seem to make this part work. Presumably, any print statements in listener.py would show up in outfile, correct? But all I get is an empty 'listenerOutput' file. Is there some extra step to make it write to the file, or am I going about this the wrong way?

Improve this question
3
  • Can you add a > listenerOutput when you're using Popen? Commented Aug 14, 2014 at 16:45
  • 1
    @TheSoundDefense - he'd also need to set shell=True, but it also won't help. His stdout trick is correct. Not waiting for it to complete is the problem. Commented Aug 14, 2014 at 16:59
  • I ended up not being able to figure it out. In fact, the moment I do this in the command line: python listener.py > listener_out & it doesn't write to anything! I still don't know why, so instead I used python's logging class in my listener to record everything and that works. And instead of using python to launch subprocesses, I used a bash script to make things simple. Still, if anyone has an idea on what went wrong, I'm all ears :] Commented Aug 16, 2014 at 0:25

2 Answers 2

Reset to default
1

Popen does not block, so it returns as soon as the subprocess begins. Being the last line of the with statement, the context manager immediately closes the output file, which prevents listener.py from writing to it.

(Update: the child still has its own open file descriptor after the with statement exits. Exactly why the child's output isn't being written to the file isn't clear, but it's likely that it is somehow exiting prematurely.)

Either wait for the process to complete:

with open('listenerOutput', 'w') as outfile: Popen(["python", "listener.py"], stdout = outfile).wait()

or skip the with statement and close the file manually later.

fh = open('listenerOutput', 'w') p = Popen(..., stdout=fh) # Do some other stuff in the meantime. p.wait() fh.close()
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

I agree with the solution, but not so much the cause. The with clause closes the file for the parent but not for the child. I suspect that the parent process exits which kills its child before it writes.
That could be, but I'm not sure how subprocess actually works. Does it pass a file descriptor to the child, or does it simply open a pipe which the child writes to, and the Popen process reads from the pipe and writes to the open file object?
It passes a file descriptor to the child which means there are 2 open file descriptors to the file. Its mostly harmless unless you are pipelining multiple commands but you could close it in the parent immediately without harm.
You're right about the file descriptor being passed; however, simply exiting the parent does not cause the child to be killed (although there may be some other way the child is being terminated prematurely).
Yeah, your right... i should have thought about that more. Perhaps listener.py is running the the background and will eventually write the file when its internal buffers finally fill enough.
|
0

The listener script is probably not flushing stdout. The console probably has auto-flush so you didn't notice before.

Putting sys.stdout.flush() after your write/print in listener.py will flush the output.

Tested with a similar Popen setup and a script that printed a message and then waited 5 seconds. Message appeared in the file after 5 seconds without flush, immediately with.

Improve this answer

1 Comment

This is useful if the data really does need to get to the file immediately. Generally, its best to let the program buffer and flush on its own for better performance.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.